flypythoncom/GPT2-Chinese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update tokenization_bert_word_level.py

Browse Source
This commit is contained in:
Duzeyao
2019-10-25 23:37:40 +08:00
parent 9b630a53fe
commit c7c3f0349e
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tokenizations/tokenization_bert_word_level.py
View File

@@ -23,7 +23,7 @@ import unicodedata
import thulac import thulac
from io import open from io import open
from pytorch_transformers.tokenization_utils import PreTrainedTokenizer from transformers.tokenization_utils import PreTrainedTokenizer
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)