Agreement, the f-measure, and reliability in information retrieval.

PubWeight™: 7.50‹?› | Rank: Top 0.1%

🔗 View Article (PMC 1090460)

Published in J Am Med Inform Assoc on January 31, 2005

Authors

George Hripcsak1, Adam S Rothschild

Author Affiliations

1: Department of Medical Informatics, Columbia University, 622 West 168th Street, VC5, New York, NY 10032, USA. hripcsak@columbia.edu

Articles citing this

Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assoc (2010) 13.18

Identifying patient smoking status from medical discharge records. J Am Med Inform Assoc (2007) 10.35

Evaluating the state-of-the-art in automatic de-identification. J Am Med Inform Assoc (2007) 9.28

BioInfer: a corpus for information extraction in the biomedical domain. BMC Bioinformatics (2007) 4.89

Community annotation experiment for ground truth generation for the i2b2 medication challenge. J Am Med Inform Assoc (2010) 3.05

Human and automated detection of high-frequency oscillations in clinical intracranial EEG recordings. Clin Neurophysiol (2007) 2.09

Identifying QT prolongation from ECG impressions using a general-purpose Natural Language Processor. Int J Med Inform (2008) 2.07

Discerning tumor status from unstructured MRI reports--completeness of information in existing reports and utility of automated natural language processing. J Digit Imaging (2009) 2.02

Conceptual knowledge acquisition in biomedicine: A methodological review. J Biomed Inform (2007) 2.02

Anaphoric relations in the clinical narrative: corpus creation. J Am Med Inform Assoc (2011) 1.91

TargetMine, an integrated data warehouse for candidate gene prioritisation and target discovery. PLoS One (2011) 1.89

NCBI disease corpus: a resource for disease name recognition and concept normalization. J Biomed Inform (2014) 1.83

The CLEF corpus: semantic annotation of clinical text. AMIA Annu Symp Proc (2007) 1.76

Semantic relations for problem-oriented medical records. Artif Intell Med (2010) 1.60

Large-scale evaluation of automated clinical note de-identification and its impact on information extraction. J Am Med Inform Assoc (2012) 1.58

Towards comprehensive syntactic and semantic annotations of the clinical narrative. J Am Med Inform Assoc (2013) 1.57

Construction of an annotated corpus to support biomedical information extraction. BMC Bioinformatics (2009) 1.55

Nominalization and alternations in biomedical language. PLoS One (2008) 1.52

Risk factors for bladder cancer: challenges of conducting a literature search using PubMed. Perspect Health Inf Manag (2011) 1.44

A comprehensive study of named entity recognition in Chinese clinical text. J Am Med Inform Assoc (2013) 1.26

Building an automated problem list based on natural language processing: lessons learned in the early phase of development. AMIA Annu Symp Proc (2008) 1.25

Computer-assisted update of a consumer health vocabulary through mining of social network data. J Med Internet Res (2011) 1.25

Combining classifiers for robust PICO element detection. BMC Med Inform Decis Mak (2010) 1.23

Building gold standard corpora for medical natural language processing tasks. AMIA Annu Symp Proc (2012) 1.17

Social tagging in the life sciences: characterizing a new metadata resource for bioinformatics. BMC Bioinformatics (2009) 1.17

Automated detection of critical results in radiology reports. J Digit Imaging (2012) 1.14

Annotating temporal information in clinical narratives. J Biomed Inform (2013) 1.14

Inductive creation of an annotation schema for manually indexing clinical conditions from emergency department reports. J Biomed Inform (2005) 1.13

Constructing a semantic predication gold standard from the biomedical literature. BMC Bioinformatics (2011) 1.12

Annotating the biomedical literature for the human variome. Database (Oxford) (2013) 1.11

Automatic prediction of rheumatoid arthritis disease activity from the electronic medical records. PLoS One (2013) 1.09

Protein structural similarity search by Ramachandran codes. BMC Bioinformatics (2007) 1.04

Preparing an annotated gold standard corpus to share with extramural investigators for de-identification research. J Biomed Inform (2014) 1.03

Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements. J Am Med Inform Assoc (2013) 1.00

Feasibility of feature-based indexing, clustering, and search of clinical trials. A case study of breast cancer trials from ClinicalTrials.gov. Methods Inf Med (2013) 0.99

Neural decoding of expressive human movement from scalp electroencephalography (EEG). Front Hum Neurosci (2014) 0.98

Evaluating the effects of machine pre-annotation and an interactive annotation interface on manual de-identification of clinical text. J Biomed Inform (2014) 0.97

Multi-topic assignment for exploratory navigation of consumer health information in NetWellness using formal concept analysis. BMC Med Inform Decis Mak (2014) 0.97

Syntactic parsing of clinical text: guideline and corpus development with handling ill-formed sentences. J Am Med Inform Assoc (2013) 0.96

Evaluation of an integrated clinical workflow for targeted next-generation sequencing of low-quality tumor DNA using a 51-gene enrichment panel. BMC Med Genomics (2014) 0.95

Inter-rater agreement in physician-coded problem lists. AMIA Annu Symp Proc (2005) 0.94

Mining FDA drug labels for medical conditions. BMC Med Inform Decis Mak (2013) 0.94

PASTE: patient-centered SMS text tagging in a medication management system. J Am Med Inform Assoc (2011) 0.94

Which statistical significance test best detects oncomiRNAs in cancer tissues? An exploratory analysis. Oncotarget (2016) 0.93

Discovering body site and severity modifiers in clinical texts. J Am Med Inform Assoc (2013) 0.93

Developing and evaluating an automated appendicitis risk stratification algorithm for pediatric patients in the emergency department. J Am Med Inform Assoc (2013) 0.92

Semantic annotation of clinical events for generating a problem list. AMIA Annu Symp Proc (2013) 0.91

Imaging informatics for consumer health: towards a radiology patient portal. J Am Med Inform Assoc (2013) 0.90

Modification of Acute Physiology and Chronic Health Evaluation II score through recalibration of risk prediction model in critical care patients of a respiratory disease referral center. Int J Crit Illn Inj Sci (2013) 0.90

Validation of an LC-MS based approach for profiling histones in chronic lymphocytic leukemia. Proteomics (2009) 0.90

A computational framework for understanding decision making through integration of basic learning rules. J Neurosci (2013) 0.88

Recent Advances in Clinical Natural Language Processing in Support of Semantic Analysis. Yearb Med Inform (2015) 0.88

Automated detection of radiology reports that document non-routine communication of critical or significant results. J Digit Imaging (2010) 0.87

SOAX: a software for quantification of 3D biopolymer networks. Sci Rep (2015) 0.85

Simultaneous-Fault Diagnosis of Gearboxes Using Probabilistic Committee Machine. Sensors (Basel) (2016) 0.84

Knowledge-driven geospatial location resolution for phylogeographic models of virus migration. Bioinformatics (2015) 0.84

Tumor information extraction in radiology reports for hepatocellular carcinoma patients. AMIA Jt Summits Transl Sci Proc (2016) 0.83

Physician inter-annotator agreement in the Quality Oncology Practice Initiative manual abstraction task. J Oncol Pract (2013) 0.83

BioCause: Annotating and analysing causality in the biomedical domain. BMC Bioinformatics (2013) 0.83

Reliability and type of consumer health documents on the World Wide Web: an annotation study. J Biomed Semantics (2011) 0.82

A study of terminology auditors' performance for UMLS semantic type assignments. J Biomed Inform (2012) 0.82

Extracting a stroke phenotype risk factor from Veteran Health Administration clinical reports: an information content analysis. J Biomed Semantics (2016) 0.81

Natural Language Processing Versus Content-Based Image Analysis for Medical Document Retrieval. J Am Soc Inf Sci Technol (2008) 0.81

Sortal anaphora resolution to enhance relation extraction from biomedical literature. BMC Bioinformatics (2016) 0.80

Cross-sectional relatedness between sentences in breast radiology reports: development of an SVM classifier and evaluation against annotations of five breast radiologists. J Digit Imaging (2013) 0.80

Homology-based prediction of interactions between proteins using Averaged One-Dependence Estimators. BMC Bioinformatics (2014) 0.80

A high-precision rule-based extraction system for expanding geospatial metadata in GenBank records. J Am Med Inform Assoc (2016) 0.80

Building an automated SOAP classifier for emergency department reports. J Biomed Inform (2011) 0.79

Bio-SCoRes: A Smorgasbord Architecture for Coreference Resolution in Biomedical Text. PLoS One (2016) 0.78

Sleep/wake estimation using only anterior tibialis electromyography data. Biomed Eng Online (2012) 0.78

Health recommender systems: concepts, requirements, technical basics and challenges. Int J Environ Res Public Health (2014) 0.78

How do multiple testing correction and spatial autocorrelation affect areal boundary analysis? Spat Spatiotemporal Epidemiol (2010) 0.78

A Method for Finding Metabolic Pathways Using Atomic Group Tracking. PLoS One (2017) 0.78

Natural language processing in the molecular imaging domain. AMIA Annu Symp Proc (2005) 0.77

Text Mining the History of Medicine. PLoS One (2016) 0.77

Is relevance relevant? User relevance ratings may not predict the impact of Internet search on decision outcomes. J Am Med Inform Assoc (2008) 0.77

CUSTOM-SEQ: a prototype for oncology rapid learning in a comprehensive EHR environment. J Am Med Inform Assoc (2016) 0.76

Visualizing the operating range of a classification system. J Am Med Inform Assoc (2012) 0.75

Imbalanced target prediction with pattern discovery on clinical data repositories. BMC Med Inform Decis Mak (2017) 0.75

Local versus global biological network alignment. Bioinformatics (2016) 0.75

Leveraging Electronic Health Care Record Information to Measure Pressure Ulcer Risk in Veterans With Spinal Cord Injury: A Longitudinal Study Protocol. JMIR Res Protoc (2017) 0.75

A vector space method to quantify agreement in qualitative data. AMIA Annu Symp Proc (2008) 0.75

Agreement between patient-reported and provider-reported choice of contraceptive method among family planning patients in New York City: implications for public health. Int J Womens Health (2013) 0.75

Effects of Sleep Deprivation on Phase Synchronization as Assessed by Wavelet Phase Coherence Analysis of Prefrontal Tissue Oxyhemoglobin Signals. PLoS One (2017) 0.75

Evaluating alignment quality between iconic language and reference terminologies using similarity metrics. BMC Med Inform Decis Mak (2014) 0.75

Annotating patient clinical records with syntactic chunks and named entities: the Harvey Corpus. Lang Resour Eval (2016) 0.75

Identifying Topics in Microblogs Using Wikipedia. PLoS One (2016) 0.75

Understanding Depressive Symptoms and Psychosocial Stressors on Twitter: A Corpus-Based Study. J Med Internet Res (2017) 0.75

Developing a knowledge base to support the annotation of ultrasound images of ectopic pregnancy. J Biomed Semantics (2017) 0.75

Normalizing acronyms and abbreviations to aid patient understanding of clinical texts: ShARe/CLEF eHealth Challenge 2013, Task 2. J Biomed Semantics (2016) 0.75

Suboptimal Clinical Documentation in Young Children with Severe Obesity at Tertiary Care Centers. Int J Pediatr (2016) 0.75

Automatic query generation using word embeddings for retrieving passages describing experimental methods. Database (Oxford) (2017) 0.75

Tremor Detection Using Parametric and Non-Parametric Spectral Estimation Methods: A Comparison with Clinical Assessment. PLoS One (2016) 0.75

Developing a cardiovascular disease risk factor annotated corpus of Chinese electronic medical records. BMC Med Inform Decis Mak (2017) 0.75

TwiMed: Twitter and PubMed Comparable Corpus of Drugs, Diseases, Symptoms, and Their Relations. JMIR Public Health Surveill (2017) 0.75

Evaluation and comparison of statistical methods for early temporal detection of outbreaks: A simulation-based study. PLoS One (2017) 0.75

Articles by these authors

Measuring agreement in medical informatics reliability studies. J Biomed Inform (2002) 5.00

Inter-patient distance metrics using SNOMED CT defining relationships. J Biomed Inform (2006) 2.12