文章要約のPegasusを動かそうとしたときに起きたエラー
huggingfaceで提供されている文章要約モデルPegasusを下記ドキュメント通り動かそうとすると
TypeError: 'NoneType’ object is not callable
のエラーが発生した．
https://huggingface.co/docs/transformers/model_doc/pegasus
調べてみると，PegasusTokenizerをAutoTokenizerに変更するとうまくいった．
下記にサンプルを掲載．正常に動作すれば，tgt_textに要約された文章が格納されているはずだ．
from transformers import PegasusForConditionalGeneration, AutoTokenizer
import torch
src_text = [
""" Hi, all. This is Joe Beninati on for Colin. Thanks for taking our questions. So given your R&D -- I'm well. Given the change in R&D and capex spending, could you detail which of the newer products are going to be a primary focus how we should think about time to market, and then finally, plans for reaching EBITDA breakeven?"""
]
model_name = 'google/pegasus-xsum'
device = 'cuda' if torch.cuda.is_available() else 'cpu'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = PegasusForConditionalGeneration.from_pretrained(model_name).to(device)
batch = tokenizer(src_text, truncation=True, padding='longest', return_tensors="pt").to(device)
translated = model.generate(**batch)
tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True)