Flair Training Sequence Labeling Models

(C) 2019 by Damir Cavar

Version: 0.2, September 2019

Download: This and various other Jupyter notebooks are available from my GitHub repo.

For the Flair tutorial 7 license and copyright restrictions, see the website below. For all the parts that I added, consider the license to be Creative Commons Attribution-ShareAlike 4.0 International License (CA BY-SA 4.0).

This tutorial is using the CoNLL-03 Named Entity Recognition data set. See this website for more details and to download an independent copy of the data set.

Sequence Labeling Model using CoNLL-03 Data

We will need the following modules:

In [ ]:
from flair.data import TaggedCorpus
from flair.data_fetcher import NLPTaskDataFetcher, NLPTask
from flair.embeddings import TokenEmbeddings, WordEmbeddings, StackedEmbeddings
from typing import List

If you want to use the CoNLL-03 corpus, you need to download it and unpack it in your Flair data and model folder. This folder should be in your home-directory and it is named .flair. Once you have downloaded the corpus, unpack it into a folder .flair/datasets/conll_03. If you do not want to use the CoNLL-03 corpus, but rather the free W-NUT 17 corpus, you can use the Flair command: NLPTaskDataFetcher.load_corpus(NLPTask.WNUT_17)

If you decide to download the CoNLL-03 corpus, use the following code. We load the corpus and down-sample it to 10% of its size:

In [ ]:
# use the CoNLL-13 corpus
# corpus: TaggedCorpus = NLPTaskDataFetcher.load_corpus(NLPTask.CONLL_03).downsample(0.1)
corpus: TaggedCorpus = NLPTaskDataFetcher.load_corpus(NLPTask.WNUT_17)
print(corpus)

Declare the tag type to be predicted:

In [ ]:
tag_type = 'ner'

Create the tag-dictionary for the tag-type:

In [ ]:
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)
print(tag_dictionary.idx2item)

Load the embeddings:

In [ ]:
embedding_types: List[TokenEmbeddings] = [

    WordEmbeddings('glove'),

    # comment in this line to use character embeddings
    # CharacterEmbeddings(),

    # comment in these lines to use flair embeddings
    # FlairEmbeddings('news-forward'),
    # FlairEmbeddings('news-backward'),
]
embeddings: StackedEmbeddings = StackedEmbeddings(embeddings=embedding_types)

Load and initialize the sequence tagger:

In [ ]:
from flair.models import SequenceTagger

tagger: SequenceTagger = SequenceTagger(hidden_size=256,
                                        embeddings=embeddings,
                                        tag_dictionary=tag_dictionary,
                                        tag_type=tag_type,
                                        use_crf=True)

Load and initialize the trainer:

In [ ]:
from flair.trainers import ModelTrainer

trainer: ModelTrainer = ModelTrainer(tagger, corpus)

If you have a GPU (otherwise maybe tweak the batch size, etc.), run the training with 150 epochs:

In [ ]:
trainer.train('resources/taggers/example-ner',
              learning_rate=0.1,
              mini_batch_size=32,
              max_epochs=150)

Plot the training curves and results:

In [ ]:
from flair.visual.training_curves import Plotter
plotter = Plotter()
plotter.plot_training_curves('resources/taggers/example-ner/loss.tsv')
plotter.plot_weights('resources/taggers/example-ner/weights.txt')