$ neleval --help
usage: neleval [-h] [--verbose] [--quiet]
{evaluate,validate-spans,list-measures,analyze,significance,confidence,prepare-tac,prepare-tac15,prepare-brat,prepare-conll-coref,compare-measures,rank-systems,plot-systems,compose-measures,to-weak,select-alternatives,weights-for-hierarchy}
...
Evaluation tools for Named Entity Linking output.
positional arguments:
{evaluate,validate-spans,list-measures,analyze,significance,confidence,prepare-tac,prepare-tac15,prepare-brat,prepare-conll-coref,compare-measures,rank-systems,plot-systems,compose-measures,to-weak,select-alternatives,weights-for-hierarchy}
evaluate Evaluate system output
validate-spans Identify duplicate, crossing and nested spans
list-measures List measures schemes available for evaluation
analyze Analyze errors
significance Test for pairwise significance between systems
confidence Calculate percentile bootstrap confidence intervals
for a system
prepare-tac Convert TAC output format for evaluation
prepare-tac15 Convert TAC 2015 KBP EL output format for evaluation
prepare-brat Convert brat format for evaluation
prepare-conll-coref
Import format from CoNLL 2011-2 coreference shared
task for evaluation
compare-measures Calculate statistics of measure distribution over
systems
rank-systems Get filenames corresponding to best-ranked systems
plot-systems Summarise system results as scatter plots
compose-measures Adds composite measures rows to evaluation output
to-weak Convert annotations to char-level for weak evaluation
select-alternatives
Handle KB ambiguity in the gold standard by modifying
it to match system
weights-for-hierarchy
Translate a hierarchy of types into a sparse matrix of
type-pair weights
optional arguments:
-h, --help show this help message and exit
--verbose
--quiet