ManTIME

ManTIME is an open-source machine learning pipeline for the extraction of temporal expressions from general domain texts.

Demo

This demo is using the model trained on human-annotated data (gold only) and today's date as utterance.
Please, do not type more than one sentence. I do not integrate a sentence splitter yet, performance could be bad.

Lack of creativity? Use this: "For nearly forty years, the United States has said categorically it would not tolerate totalitarian rule in its own backyard."

Download

Source code

ManTIME source code is hosted by GitHub.

CRF++ models

The pipeline uses a CRFs model for the identification phase. You can use one of the following pre-trained models, depending on the training set:

Gold and Silver data
Size: 517.3Mb; MD5: 3cd9c9d6a7c82fc4f08f5f3547a1b1bd, download
Gold data only
Size: 85.6Mb; MD5: 5f7bb9a80bc0b684d27d8b512b26e39b, download
Silver data only
Size: 464.2Mb, MD5: f13fdfba324e64adbbd6ce0d73037b5b, download

Related publications

ManTIME: Temporal expression identification and normalization in TempEval-3 challenge
M. Filannino, G. Brown, G. Nenadic
Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval 2013)

Temporal expression normalisation in natural language texts
M. Filannino
CoRR, abs/1206.2010, 2012

Statistical validation

The following tables show the results of the model selection procedure and the analysis of statistical significant for the application of the post-processing pipeline. The figures are F1-scores.

Model selection

# shuffling # fold Models
Model 1 Model 2 Model 3 Model 4
Shuffling 0Fold 083.6083.7884.2283.27
Fold 183.6783.7383.5083.38
Fold 284.3884.7884.4484.49
Fold 383.0182.8483.0282.81
Fold 483.5083.4983.8583.54
Fold 583.3083.1283.5882.60
Fold 684.2984.4484.7783.82
Fold 782.9883.4383.3882.66
Fold 881.7882.2382.2381.95
Fold 984.9184.4584.9283.81
Shuffling 1Fold 083.2283.2183.3583.50
Fold 182.9483.1283.2082.59
Fold 284.9784.4984.6683.81
Fold 382.8582.3883.0783.28
Fold 483.6683.7583.5583.38
Fold 584.6484.6684.5984.37
Fold 683.4483.2083.4082.26
Fold 784.5484.1284.8383.56
Fold 881.6781.9881.8581.73
Fold 983.9683.9283.4383.26
Shuffling 2Fold 083.2083.6683.2282.78
Fold 184.3584.3784.8983.64
Fold 283.9683.5584.1983.33
Fold 383.2983.8183.4783.66
Fold 481.8882.0082.0581.42
Fold 583.4083.3583.5882.45
Fold 683.6783.3383.3783.08
Fold 784.3785.0084.7483.98
Fold 882.8782.6483.0282.89
Fold 984.7184.4984.4083.74
Shuffling 3Fold 084.4184.5284.7783.52
Fold 183.4283.0883.2882.50
Fold 284.5984.5884.2584.04
Fold 383.4684.8483.9483.41
Fold 481.8382.0481.9881.80
Fold 584.6684.2184.6683.64
Fold 683.3383.2683.2082.62
Fold 782.8582.8482.8082.98
Fold 884.0583.9683.6583.25
Fold 983.7283.9983.5483.06
Shuffling 4Fold 082.1082.0182.0881.56
Fold 184.6884.8784.1484.46
Fold 284.6284.6084.6683.78
Fold 384.0983.8284.2883.19
Fold 483.5983.3083.2082.53
Fold 583.3283.0983.4482.33
Fold 682.9482.9482.6183.07
Fold 783.6283.2283.6583.38
Fold 884.3984.1484.5083.88
Fold 983.3183.3883.4883.29
Mean83.599883.600283.657683.1466
STD0.84327711090.82597002960.82404319280.7256035067
Results of the 5x10-fold cross-validation with mean and standard deviations.
Differences among models using 5x10-fold cross-validation

Post-processing pipeline

# model # fold Post-processing pipeline application
CRFs only CRFs + pipeline
Model 1Fold 083.6085.43
Fold 183.6786.84
Fold 284.3886.11
Fold 383.0184.65
Fold 483.5086.15
Fold 583.3085.92
Fold 684.2986.17
Fold 782.9886.10
Fold 881.7884.98
Fold 984.9186.70
Model 2Fold 083.7885.02
Fold 183.7387.22
Fold 284.7886.29
Fold 382.8484.22
Fold 483.4985.75
Fold 583.1284.67
Fold 684.4486.34
Fold 783.4385.96
Fold 882.2385.36
Fold 984.4586.83
Model 3Fold 084.2285.34
Fold 183.5086.96
Fold 284.4485.74
Fold 383.0284.60
Fold 483.8586.42
Fold 583.5885.79
Fold 684.7786.27
Fold 783.3885.87
Fold 882.2385.39
Fold 984.9287.05
Model 4Fold 083.2785.00
Fold 183.3886.25
Fold 284.4986.31
Fold 382.8185.42
Fold 483.5486.79
Fold 582.6085.02
Fold 683.8285.80
Fold 782.6684.94
Fold 881.9584.87
Fold 983.8186.35
Mean83.5487585.82225
STD0.697060.95457
Results of the 10-fold cross-validation with mean and standard deviations.
Differences among architectures using 10-fold cross-validation.