Preliminary MED 2014 Scores

• Metadata Generation Summary
  
  • Detailed AH Training
    
  • Eval and Hardware Description
    
  • Semantic and Event Query Generation Times
    
  • CDR Info
    
  • Event Query Generatio, Semantic Query Generation and Event Search Hardware Descriptions
    
• R0 Metric ID Results
  

• Pre-Specified Systems

  
  

  			Scores on MED14Eval Full (200K videos)						Scores on MED14Eval Sub (32K videos)
  		MED MAP	MED MAP						MED MAP
  			noPRF				PRF		noPRF				PRF
  		System Description	SQ	000Ex	010Ex	100Ex	000Ex	010Ex	SQ	000Ex	010Ex	100Ex	000Ex	010Ex
  Teams that processed MED14Full	ATTLabs	MED14Full_PS_010Ex_2.txt			5.3	11.1					7.2	14.5
  	AXES	MED14Full_PS_010Ex_2.txt			12.7	28.5					18.9	36.6
  	Aurora	MED14Full_PS_000Ex_3.txt	3.5	3.5	* 13.5	* 25.7			6.7	6.7	* 19.6	* 32.5
  	BBNVISER	MED14Full_PS_000Ex_2.txt	5.3	5.7	18.0	29.8			8.8	10.0	24.8	36.9
  	CMU	MED14Full_PS_000Ex_2.txt	14.9	15.5	19.4	32.3	18.1	20.3	20.0	21.2	25.7	39.6	24.1	26.2
  	Fudan	MED14Full_PS_010Ex_2.txt			+ 10.7	+ 22.1					+ 15.0	+ 29.2
  	INRIA-LIM-VocR	MED14Full_PS_010Ex_2.txt			14.0	29.7					20.0	37.9
  	MediaMill	MED14Full_PS_010Ex_2.txt	3.6		15.1	24.3			7.3		20.6	29.8
  	NII	MED14Full_PS_010Ex_1.txt			8.0	21.6					11.6	28.1
  	Sesame	MED14Full_PS_010Ex_2.txt	5.1		18.3	29.9			8.6		23.7	38.1
  	TokyoTech	MED14Full_PS_010Ex_2.txt			8.0	21.9					13.4	29.2
  	VIREO	MED14Full_PS_000Ex_2.txt	4.0	5.2	12.4	15.8			6.0	7.7	18.5	23.1
  Teams that processed MED14Sub	ITI_CERTH	MED14Sub_PS_010Ex_2.txt									15.1	30.3
  	KU-ISPL	MED14Sub_PS_010Ex_5.txt									+ 2.4	+ 4.7
  	MCIS	MED14Sub_PS_010Ex_4.txt									15.6
  	MIC	MED14Sub_PS_000Ex_4.txt								* 0.2	* 0.6	* 2.7
  	ORAND	MED14Sub_PS_010Ex_2.txt									1.2	5.0

  
  
  

• Pre-Adjudicated Ad-Hoc Systems

  
  

  			Scores on MED14Eval Full (200K videos)							Scores on MED14Eval Sub (32K videos)
  		MED MAP	MED MAP							MED MAP
  			noPRF				PRF			noPRF				PRF
  		System Description	SQ	000Ex	010Ex	100Ex	SQ	000Ex	010Ex	SQ	000Ex	010Ex	100Ex	SQ	000Ex	010Ex
  Teams that processed MED14Full	ATTLabs	MED14Full_AH_010Ex_2.txt			0.7	6.8						3.4	17.5
  	AXES	MED14Full_AH_010Ex_2.txt			12.2	20.9						22.1	32.0
  	Aurora	MED14Full_AH_000Ex_4.txt	1.9	* 1.9	* 8.5	* 16.7	0.8	* 1.4		4.2	* 4.2	* 15.6	* 27.7	2.2	* 3.4
  	BBNVISER	MED14Full_AH_000Ex_2.txt	3.1	3.7	10.9	22.6				7.0	8.2	20.2	34.2
  	CMU	MED14Full_AH_000Ex_2.txt	8.0	8.0	12.7	24.0		12.2	14.5	16.1	16.1	21.0	36.5		22.1	23.3
  	Fudan	MED14Full_AH_010Ex_2.txt			7.4	15.6						14.2	26.9
  	INRIA-LIM-VocR	MED14Full_AH_010Ex_2.txt			12.7	21.8						22.8	33.0
  	MediaMill	MED14Full_AH_010Ex_2.txt	1.8		9.0	18.6				4.9		16.3	30.5
  	NII	MED14Full_AH_010Ex_1.txt			4.9	14.3						11.1	23.7
  	Sesame	MED14Full_AH_010Ex_4.txt	1.7		* 12.3	* 23.0				4.7		* 21.4	* 36.9
  	TokyoTech	MED14Full_AH_010Ex_2.txt			6.5	17.2						12.6	26.9
  	VIREO	MED14Full_AH_000Ex_2.txt	1.6	2.6	7.7					4.3	5.7	15.1
  Teams that processed MED14Sub	ITI_CERTH	MED14Sub_AH_010Ex_2.txt										16.2	30.2
  	KU-ISPL	MED14Sub_AH_010Ex_4.txt										+ 2.0	+ 2.7
  	MCIS	MED14Sub_AH_010Ex_2.txt										14.3
  	MIC	MED14Sub_AH_000Ex_2.txt									0.4	0.9	3.1
  	ORAND	MED14Sub_AH_010Ex_2.txt										4.3	10.0

  
  * - Debugged submission
  + - Late Submission

Update history:

• 2014.09.08 - Initial release
  
• 2014.09.17 - Corrected NII scoring to include scores for MED14Full. Added Processing time measurements. Added System Descriptions.
  
• 2014.10.10
  
• The evaluated clip set was corrected to exclude previously banned clips.
  
• Debugged Sesame AH Results were included.
  
• 2014.10.20
  
• Added the EQG, ES, and EQG Hardware Descriptions.
• Debugged Sesame AH Results were included.
• BBNVISER Signal MD Size corrected.