yidao620c/python3-cookbook
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: fix an indent error

Browse Source
This commit is contained in:
noahsss
2017-09-24 21:33:29 -05:00
committed by GitHub
parent 71a80a8f42
commit 22258339e1
1 changed files with 6 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
source/c02/p18_tokenizing_text.rst
View File

@@ -22,7 +22,7 @@
.. code-block:: python .. code-block:: python
tokens = [('NAME', 'foo'), ('EQ','='), ('NUM', '23'), ('PLUS','+'), tokens = [('NAME', 'foo'), ('EQ','='), ('NUM', '23'), ('PLUS','+'),
('NUM', '42'), ('TIMES', '*'), ('NUM', '10')] ('NUM', '42'), ('TIMES', '*'), ('NUM', '10')]
为了执行这样的切分，第一步就是像下面这样利用命名捕获组的正则表达式来定义所有可能的令牌，包括空格： 为了执行这样的切分，第一步就是像下面这样利用命名捕获组的正则表达式来定义所有可能的令牌，包括空格：
@@ -76,10 +76,10 @@
.. code-block:: python .. code-block:: python
def generate_tokens(pat, text): def generate_tokens(pat, text):
Token = namedtuple('Token', ['type', 'value']) Token = namedtuple('Token', ['type', 'value'])
scanner = pat.scanner(text) scanner = pat.scanner(text)
for m in iter(scanner.match, None): for m in iter(scanner.match, None):
yield Token(m.lastgroup, m.group()) yield Token(m.lastgroup, m.group())
# Example use # Example use
for tok in generate_tokens(master_pat, 'foo = 42'): for tok in generate_tokens(master_pat, 'foo = 42'):
@@ -97,7 +97,7 @@
.. code-block:: python .. code-block:: python
tokens = (tok for tok in generate_tokens(master_pat, text) tokens = (tok for tok in generate_tokens(master_pat, text)
if tok.type != 'WS') if tok.type != 'WS')
for tok in tokens: for tok in tokens:
print(tok) print(tok)