Monday, August 28, 2017

GSoC 2017 - Final Report

Patient Matching 2.0

Student               -    Lahiru Jayathilake 
Primary Mentor   -    Burke Mamlin
Backup Mentor   -    Shaun Grannis

Project Wiki        -    Patient Matching 2.0
TALK Thread      -    OpenMRS TALK Thread Patient Matching 2.0
GitHub  Fork       -    Lahiru-J/patient-matching

What is Patient Matching 2.0

Patient Matching Module is an application where it tries to identify records that belong to the same patient among different number of data sources. This module is significant because in real world it has to be dealt with erroneous data. For example, a patient’s name can be misspelled. Benefit of this module is by linkage of records it would be easier for a patient to visit a hospital without carrying hard copies of test results whether or not the tests were done at the same hospital.

Summary of the Work

✔︎ Incremental Patient Matching

To give a little introduction on what incremental patient matching is, it is a method of identifying duplicate patients very efficiently. 
In a real world scenario, the requirement would be to match thousands of patients with each other. Once this matching process continues it would be very time consuming since all of the patients are compared with each of them regardless of the fact that two or three new patients are added/updated. Incremental patient matching is used as a suggestion to save time.

This task was my primary goal and I have successfully completed it.
Following are the commits & pull requests related to this task. Note that commits related to a branch are squashed according to the OpenMRS convention.


PTM-82 -  

PTM-83 -

PTM-84 -

PTM-85 -

PTM-86 -

PTM-89 -

Pull Requests

PTM-82: Functionality to load patients considering the date created and date changed

PTM-83: Save and update incremental patient matching report to the database

PTM-84: Functionality to support two datasources

PTM-85: Functionality to select or deselect incremental match

PTM-86 : Remove matching pairs from the report when patients are updated

PTM-89 : Ignore voided patients when running a patient match

✔︎ Merge Patients

If some of the patients in a group supposed to be the same then the user can merge those patients making they will not appear again in a patient matching report. 

PTM-87 -

Pull Request
PTM-87 - Functionality to Merge Patients in the report

✔︎ Exclude Non-Matching Patients

There can be some scenarios where the module results some of patients to be same but in real life those patients are related to totally different people. If this happens Patient Matching 2.0 project provides a functionality to eliminate such records without them repeatedly appearing on a patient matching report.

PTM-88 -

Pull Request
PTM-88 : Functionality to exclude non-matching patients

Blog Posts

Blog Post
Week 12
Week 11
Week 10
Week 9
Week 8
Week 7
Week 6
Week 5
Week 4
Week 3
Week 2
Week 1


Throughout this summer I learned a lot. Whenever there were problems related to the project I had a great support from my mentor and the community. Participating in daily scrums showed me how good this OpenMRS community is. 
My mentors are the best. They gave me guidance at the first place, conducting many video conferences because of that I could carry out this project with a sound knowledge. Burke, Shaun thank you very much for your support throughout this summer. I feel really grateful to these mentors.

Monday, August 21, 2017

Week 12 : Exclude Non Matching Patients

In this week I have been working on the PTM-88 which is to exclude set of non-matching patient records from the report generation process. 

The main reason to carryout this task is, the match process generates a list of highly likely matches (probable duplicates) for human review. The human reviewer will declare true matches and non-matches from this list. Currently, the Patient Matching module repeatedly presents all likely matches each time it runs, without being informed by the human reviewer information learned from prior runs. 

User Interface,

You can exclude the patients by selecting them and clicking the button Exclude Patients.

Sunday, August 13, 2017

Week 11 : Merge Patients

Once after generating the report if user thinks some patients in a particular group are the same then there should be a functionality to merge them. In this week I have completed it successfully. 

You can merge the patients by selecting them and clicking the button merge.

Once the set of patients are merged the report will be look like as follows, 

This task was carried out under the PTM-87 and the pull request can be found here.

Sunday, August 6, 2017

Week 10 : Nice Report :)

In this week I have been doing some enhancements to the patient matching report. The reason I had to do that when the patients are updated in a way that it would affect to the  patient matching report, then the report too should be updated. 

Consider the following patient matching report.

If the patient 13(Unique ID) is updated in a way that it will no longer exhibits any matching properties with patient 12, then the group 4 should be removed from the report. But if patient 23 is updated then only the patient 23 should be removed from the report not every record in the group 3.

The code that I written for the above purpose looks like as follows,

Sunday, July 30, 2017

Week 9 : Incremental Patient Match

Good news, everything which is needed to perform an incremental match has been completed. In this week I did complete the PTM-85 which is mainly about taking the user's decision whether to perform the patient match as an incremental match.

If the user selects the match as an incremental then the time takes to perform the match will be less compared to a normal patient match. This performance is achieved because the data set to be compared is small. The data set contains only the newly added patients and updated patients after the last execution date of the report.

What if the user selects an incremental match and a patient match has not been done under the particular strategy? 
For the first time every patient record is compared with every other record. This process might take considerable time depending on the size of the patient records.

Despite whether there is an already existing report user can perform a full patient match for a particular strategy. 

Pull request for PTM-85 : 

The web page looks like as follows,

Incremental Patient Match

After all of theses changes this is how it looks like.

1. Run a report with the configuration name "test1"

A report will be created adding the incremental-report text to the configuration name.

2. This is how it looks like when a patient match is performed at the first time

3. Added a patient which shows a match with an existing record

4. Run the report again (Incremental patient matching)

The same incremental-report-test1 report will be updated.

5. New patient is added which do not exhibit any matching property with existing records

Then there will be no changes in the report.

6. Add another patient

7. Run the patient match then the report will be updated as the following image

8. Update an existing patient in a way that it will show matching properties with existing records

The updated patient will be added to the same group in the report.

Here is the link for my mid term presentation : 

Sunday, July 23, 2017

Week 8: Game is almost done

I have completed almost all of the tasks that I have included in the project plan and made the third pull request. The pull request is for PTM-84 and after doing this task, patient matching module supports to match patients incrementally.    

To complete this task I had to change the class in the patient matching module. Those changes can be found here.

What I have done up to now?
So far I have completed,

  • PTM-82 - Load patients for the incremental matching
  • PTM-83 - Generate and save reports in incremental patient matching process
  • PTM-84 - Perform patient match with two datasources

Sunday, July 16, 2017

Week 7: Game of Codes ;)

The tasks that I have aforementioned in my 5th week's blog post, have already been completed. The main target in PTM-84 task is to make patient matching module to deal with two datasources. The importance of this process has been mentioned in my 5th week blog post

I had to change 6 methods in Methods are listed below. 
  1. InitScratchTable
  2. CreRanSamAnalyzer
  3. CreAnalFormPairs
  4. CrePairdataSourAnalyzer
  5. ScoringData
  6. CreatingReport
Every method indicated above should support for both deduplication as well as for two datasources. Deduplication process is needed in the incremental patient matching process at the first time since every patient record is being matched with every other record. Not only that if the user specifically indicates to run the patient match for all the records, deduplication process is the one should carry out the task.

Power of the incremental patient matching comes with two datasources. One datasource is comprised with all the patients while the other datasource contains only the patients who are added or changed after the last execution date of the report.

What I have done ?

The changes that I have done for the patient matching module to get this work done can be found here.