fastNLP.api

fastNLP.api.api

fastNLP.api.converter

fastNLP.api.model_zoo

fastNLP.api.model_zoo.load_url(url, model_dir=None, map_location=None, progress=True)

Loads the Torch serialized object at the given URL.

If the object is already present in model_dir, it’s deserialized and returned. The filename part of the URL should follow the naming convention filename-<sha256>.ext where <sha256> is the first eight or more digits of the SHA256 hash of the contents of the file. The hash is used to ensure unique names and to verify the contents of the file.

The default value of model_dir is $TORCH_HOME/models where $TORCH_HOME defaults to ~/.torch. The default directory can be overridden with the $TORCH_MODEL_ZOO environment variable.

Args:

url (string): URL of the object to download model_dir (string, optional): directory in which to save the object map_location (optional): a function or a dict specifying how to remap storage locations (see torch.load) progress (bool, optional): whether or not to display a progress bar to stderr

Example:

# >>> state_dict = model_zoo.load_url(‘https://s3.amazonaws.com/pytorch/models/resnet18-5c106cde.pth’)

fastNLP.api.pipeline

class fastNLP.api.pipeline.Pipeline(processors=None)

Pipeline takes a DataSet object as input, runs multiple processors sequentially, and outputs a DataSet object.

fastNLP.api.processor

class fastNLP.api.processor.FullSpaceToHalfSpaceProcessor(field_name, change_alpha=True, change_digit=True, change_punctuation=True, change_space=True)

全角转半角，以字符为处理单元

class fastNLP.api.processor.Index2WordProcessor(vocab, field_name, new_added_field_name)

将DataSet中某个为index的field根据vocab转换为str

class fastNLP.api.processor.IndexerProcessor(vocab, field_name, new_added_field_name, delete_old_field=False, is_input=True)

给定一个vocabulary , 将指定field转换为index形式。指定field应该是一维的list，比如

[‘我’, ‘是’, xxx]

class fastNLP.api.processor.Num2TagProcessor(tag, field_name, new_added_field_name=None)

将一句话中的数字转换为某个tag。

class fastNLP.api.processor.PreAppendProcessor(data, field_name, new_added_field_name=None)

向某个field的起始增加data(应该为str类型)。该field需要为list类型。即新增的field为

[data] + instance[field_name]

class fastNLP.api.processor.SeqLenProcessor(field_name, new_added_field_name='seq_lens', is_input=True)

根据某个field新增一个sequence length的field。取该field的第一维

class fastNLP.api.processor.SliceProcessor(start, end, step, field_name, new_added_field_name=None)

从某个field中只取部分内容。等价于instance[field_name][start:end:step]

class fastNLP.api.processor.VocabProcessor(field_name, min_freq=1, max_size=None)

传入若干个DataSet以建立vocabulary。