Sunday, July 30, 2017

Week 9 : Incremental Patient Match

Good news, everything which is needed to perform an incremental match has been completed. In this week I did complete the PTM-85 which is mainly about taking the user's decision whether to perform the patient match as an incremental match.


If the user selects the match as an incremental then the time takes to perform the match will be less compared to a normal patient match. This performance is achieved because the data set to be compared is small. The data set contains only the newly added patients and updated patients after the last execution date of the report.


What if the user selects an incremental match and a patient match has not been done under the particular strategy? 
For the first time every patient record is compared with every other record. This process might take considerable time depending on the size of the patient records.

Despite whether there is an already existing report user can perform a full patient match for a particular strategy. 

Pull request for PTM-85 : https://github.com/openmrs/openmrs-module-patientmatching/pull/37 

The web page looks like as follows,



Incremental Patient Match

After all of theses changes this is how it looks like.

1. Run a report with the configuration name "test1"



A report will be created adding the incremental-report text to the configuration name.


2. This is how it looks like when a patient match is performed at the first time



3. Added a patient which shows a match with an existing record



4. Run the report again (Incremental patient matching)



The same incremental-report-test1 report will be updated.

5. New patient is added which do not exhibit any matching property with existing records



Then there will be no changes in the report.

6. Add another patient



7. Run the patient match then the report will be updated as the following image




8. Update an existing patient in a way that it will show matching properties with existing records


The updated patient will be added to the same group in the report.


Here is the link for my mid term presentation : https://www.youtube.com/watch?v=j-m9kDQmdz0&t=5s 

Sunday, July 23, 2017

Week 8: Game is almost done

I have completed almost all of the tasks that I have included in the project plan and made the third pull request. The pull request is for PTM-84 and after doing this task, patient matching module supports to match patients incrementally.    

To complete this task I had to change the MatchingReportUtils.java class in the patient matching module. Those changes can be found here.

What I have done up to now?
So far I have completed,

  • PTM-82 - Load patients for the incremental matching
  • PTM-83 - Generate and save reports in incremental patient matching process
  • PTM-84 - Perform patient match with two datasources


Sunday, July 16, 2017

Week 7: Game of Codes ;)

The tasks that I have aforementioned in my 5th week's blog post, have already been completed. The main target in PTM-84 task is to make patient matching module to deal with two datasources. The importance of this process has been mentioned in my 5th week blog post

I had to change 6 methods in MatchingReportUtils.java. Methods are listed below. 
  1. InitScratchTable
  2. CreRanSamAnalyzer
  3. CreAnalFormPairs
  4. CrePairdataSourAnalyzer
  5. ScoringData
  6. CreatingReport
Every method indicated above should support for both deduplication as well as for two datasources. Deduplication process is needed in the incremental patient matching process at the first time since every patient record is being matched with every other record. Not only that if the user specifically indicates to run the patient match for all the records, deduplication process is the one should carry out the task.

Power of the incremental patient matching comes with two datasources. One datasource is comprised with all the patients while the other datasource contains only the patients who are added or changed after the last execution date of the report.

What I have done ?

The changes that I have done for the patient matching module to get this work done can be found here.


Monday, July 10, 2017

Week 6: One more datasource!

According to my last week post I need to change few methods in patient matching module, in order to have a patient match with two datasources. So I started with following methods in MatchingReportUtils.java
  1. InitScratchTable
  2. CreRanSamAnalyzer
  3. CreAnalFormPairs
  4. CrePairdataSourAnalyzer
  5. ScoringData
  6. CreatingReport
Following code shows how I changed the code to support for both deduplication as well as for two datasources.

This code segment was added inside the InitScratchTable,

CreRanSamAnalyzer method was changed as follows,

Saturday, July 1, 2017

Week 5: Important Work

In this week I have been working on PTM-84. As I mentioned in OpenMRS talks the current version does not support to match patients by taking two datasources. That means it only supported to match patients by deduplication. 

What is required to do?
According to my ultimate goal Patient Matching 2.0 module must support for the incremental patient matching. For that it is required to,
  • Fetch patients considering the date created or date changed with the report date
  • Fetch all the patients (except the patients that have been already fetched)
and perform the match with two data sources.

Why this is necessary?
The current version is not an efficient method for implementations having huge set of records.

For instance, if we have 10,000 patients in our system and we need to match the patients using first name and the date of birth. Goal is to check for the duplicates among them. If we compare all patients to all the others that is roughly 50 million comparisons ( 10,000 x (10,000 - 1) / 2 ). After couple of days if we run the same match where 90 patients have been added and 10 updated, with the current version it would still carry out the same method of comparison and this time it would be about 51 million comparisons!
The current goal is to perform comparisons only for the added and updated records for that particular match. If we have this sort of an amazing method rather than 51 million of comparisons it would result in only about 1 million [(100 x 99 / 2) + (100 x 9990)]. 

In order to do that we should consider two datasources without that it is just matching updated or added patients with themselves. (deduplication)

What I have done ?
These are the things I did, I had to change couple of methods in MatchingReportUtils.java
The methods are
  1. InitScratchTable
  2. CreRanSamAnalyzer
  3. CreAnalFormPairs
  4. CrePairdataSourAnalyzer
  5. ScoringData
  6. CreatingReport
In order to support for two datasources rather than deduplication.