Patient Matching Algorithm Challenge

213 views Leave a comment

The idea of a Patient Matching Algorithm Challenge is to move about larger clarity and information on a opening of existent studious relating algorithms, coax a adoption of opening metrics for studious information relating algorithm vendors, and definitely impact other aspects of studious relating such as deduplication and joining to clinical data. Participants will be supposing a information set and will have their answers evaluated and scored opposite a master key. Up to 6 money prizes will be awarded with a sum purse of adult to $75,000.00.

Challenge Summary

This Challenge uses a vast exam information set, supposing by ONC, opposite that participants contingency run their algorithms and yield their formula for evaluation. A tiny set of loyal review pairs (that have been total and accurate by primer review) exist within a vast information set and will offer as a “answer key” opposite that participants’ submissions will be scored.

Participants will clear and download a exam information set during a time of registration. Participants will afterwards run their algorithms and contention their formula to a scoring server on a this website. These submissions will accept opening scores and might seem on a Challenge leaderboard.  Upon submitting results, participants will accept design analysis metrics (F-scores) that can be used to beam complement improvements; a sum of 100 re-runs will be allowed.  Up to 6 participants will be comparison as winners for this Challenge and awarded money prizes.  Top prizes will be awarded to participants’ algorithms that beget a top F-Score(s). Additionally, algorithms with a best recall, best precision, and best initial F-Score run opening will also accept a money prize.

Background Information

In late 2015, a Office of a National Coordinator for Health Information Technology (ONC) published “Connecting Health and Care for a Nation: A Shared Nationwide Interoperability Roadmap” (the Roadmap). The final Roadmap reflects a total contributions of dozens of experts and hundreds of open comments perceived during a drafting phase. The Roadmap includes “Section L,” that was privately framed to simulate a hurdles health caring faces with honour to accurate particular information matching. This territory highlights matching’s altogether significance to interoperability and a nation’s health IT infrastructure. Indeed, health caring providers contingency be means to share studious health information and accurately review a studious to his or her information from a opposite provider in sequence for many expected interoperability advantages to be realized. Conversely, relating mistakes can minister toward inauspicious events, compromised reserve and privacy, and increasing health caring costs due to repeat tests, and other factors. The cost to manually scold incompatible studious annals is estimated to be $60 per record not including a intensity mistreat that could be caused due to a studious receiving a wrong diagnosis and intensity authorised fees.

Given a concrete impacts bad studious relating can have on caring delivery, it is critical for organizations to be means to quantify their studious relating algorithm’s opening and review a formula to attention customary benchmarks and opening metrics. To date, a deficiency of such benchmarks as good as a baseline for opposite use cases has done it formidable to make advances in studious matching. Reports such as ONC’s Patient Identification and Matching Final Report list studious review rates in a operation of 50%-Mid 90%.

Every studious relating algorithm has blind spots and there are methods to calculate a opening of a studious relating algorithm and assistance brand these blind spots. This is achieved by giving an algorithm a famous information set in sequence to see how many of a famous linkages a algorithm can rightly identify. Matching algorithms can make dual forms of errors. The initial blunder is a disaster to find a relating span (often referred to as a “false negative”), that is totalled by “inverse recall” in a margin of information retrieval. The second form of blunder is a record that is matched when it should not be (often referred to as a “false positive”), that totalled by a metric famous as “precision.”  The weighted normal of pointing and remember generates a final metric impending to this Challenge is famous as “F-Score.”

Important Note

The Patient Matching Challenge non-stop on 6/12/17 during noon EST.

Challenge Timeline

  • Announcement of Challenge: Apr 28, 2017
  • Registration Period Begins May 10, 2017
  • Submission Period Begins: Upon accessibility of exam data
  • Submission Period Ends: September 12, 2017
  • Winners Notified: 1 week from a finish of acquiescence period
  • Winners Announced: 1 week from leader presentation date


On May 10May 17, and May 24 online webinars were held. The available webinar and PowerPoint slides can be found below.

Date: May 10, 2017

Recorded Webinar Link
PowerPoint Slides

Date: May 17, 2017

Recorded Webinar Link
PowerPoint Slides

Date: May 24, 2017

Recorded Webinar Link
PowerPoint Slides

Challenge Requirements

The studious relating Challenge website will control a submissions and yield a scoring formula behind to a Participant. The answer pivotal can be submitted in possibly CSV, XML, or JSON files. In sequence for a acquiescence to be authorised to win this Challenge, it contingency accommodate a following requirements:

  • No HHS or ONC logo – The product contingency not use HHS’ or ONC’s logos or central seals and contingency not explain endorsement.
  • A product might be unfit if it fails to duty as voiced in a outline supposing by a Submitter, or if it provides false or deficient information.
  • Submissions contingency be giveaway of malware. Submitter agrees that ONC might control contrast on a product to establish either malware or other confidence threats might be present. ONC might invalidate a acquiescence if, in ONC’s judgment, it might repairs supervision or others’ apparatus or handling environment.


Highest F-Score

  • First Place: $25,000
  • Second Place: $20,000
  • Third Place $15,000

Best in Category Supplemental Prizes (1 esteem for any difficulty during $5,000):

  • Best pointing (with Recall = .9)
  • Best remember (with Precision = .9)
  • Best initial F-Score run

Total Prize Purse: Up to $75,000


Comment this news or article