neleval list-measures

List measures schemes available for evaluation

Usage summary

$ neleval list-measures --help
usage: neleval list-measures [-h] [-m NAME]

List measures schemes available for evaluation

optional arguments:
  -h, --help            show this help message and exit
  -m NAME, --measure NAME
                        Which measures to use: specify a name (or group name)
                        from the list-measures command. This flag may be
                        repeated.

List all predefined measures

$ neleval list-measures
The following lists possible values for --measure (-m) in evaluate,
confidence and significance. The name from each row or the name of a
group may be used.

Name                       	Aggregate   	Filter   	Key Fields    	In groups
===========================	============	=========	==============	=======================================================
b_cubed                    	b_cubed     	None     	span          	all, all-coref, luo, tac11, tac14
b_cubed_plus               	b_cubed     	None     	span+kbid     	all, all-coref, tac11, tac14
entity_ceaf                	entity_ceaf 	None     	span          	all, all-coref, luo, tmp
entity_match               	sets        	is_linked	docid+kbid    	all, all-tagging, cornolti, hachey
mention_ceaf               	mention_ceaf	None     	span          	all, all-coref, luo, tac14, tmp
mention_ceaf_plus          	mention_ceaf	None     	span+kbid     	all, all-coref
muc                        	muc         	None     	span          	all, all-coref, luo
pairwise                   	pairwise    	None     	span          	all, all-coref, tmp
strong_all_match           	sets        	None     	span+kbid     	all, all-tagging, tac09, tac11, tac14
strong_link_match          	sets        	is_linked	span+kbid     	all, all-tagging, cornolti, hachey, tac09, tac11, tac14
strong_linked_mention_match	sets        	is_linked	span          	all, all-tagging, cornolti, hachey
strong_mention_match       	sets        	None     	span          	all, all-tagging, hachey, tac14
strong_nil_match           	sets        	is_nil   	span          	all, all-tagging, tac09, tac11, tac14
strong_typed_all_match     	sets        	None     	span+type+kbid	all, all-tagging, tac14
strong_typed_link_match    	sets        	is_linked	span+type+kbid	all, all-tagging
strong_typed_mention_match 	sets        	None     	span+type     	all, all-tagging, tac14
strong_typed_nil_match     	sets        	is_nil   	span+type     	all, all-tagging
typed_mention_ceaf         	mention_ceaf	None     	span+type     	all, all-coref, tac14
typed_mention_ceaf_plus    	mention_ceaf	None     	span+type+kbid	all, all-coref

Default evaluation group: all

In all measures, a set of tuples corresponding to Key Fields is
produced from annotations matching Filter. Aggregation with `sets'
compares gold and predicted tuple sets directly; coreference
aggregates compare tuples clustered by their assigned entity ID.

A measure may be specified explicitly. Thus:
  strong_all_match
may be entered as
  sets:None:span+kbid

Available aggregates are:
- non-clustering: overlap-maxmax, overlap-maxsum, overlap-summax, overlap-sumsum, sets
- clustering: b_cubed, entity_ceaf, mention_ceaf, muc, pairwise, pairwise_negative

Available filter and key fields: candidates, docid, eid, end,
is_first, is_linked, is_nil, kbid, link, score, span, start, type.

More fields can be stored dynamically by entering a candidate's type
as a JSON key-value mapping.