Sunday, June 25, 2017

Week 4: More Work!

According to my project plan, the tasks were planned to do in the first phase have already been completed.
Within this week I did complete the task PTM-82 and have already made the pull request

Saving the report to a persistent storage is more important in this incremental process as any run after that depends on the previous report's properties. Task PTM-83 was created targeting that very important requirements. I have almost completed this task in this week.

Things I did to get the work done in PTM-83

Concerning on the previous report, there was need for create a method to update the properties of the new report.This is not just updating an object. What made me to say this? Consider the following database table,

patientmatching_matchingset

In the above table all the pending matches are going to be shown as long as user mark them as accepted or rejected. So what happens if the user runs a strategy which already has a report and set of records in the above table? 

Let me give you an example suppose the user runs a strategy which already has a report (take report ID as 48) according to the above table it has 13 records (from set_id 125 to 137). According to the process of incremental matching it considers only the patients whose record has changed or patients who are newly added to the system after the last ran date of the above report. Let's say the match has completed and there can be two cases,

1. A patient record in a matching pair might already exists in the above table. (Take the new patient as PatientA and the it showed a match with 126th set_id)

2. Non of the patient records in the matching pairs are not in the above table.

If the match process comes across the 2nd possibility we do not have to concern more it is just an update of the table but issue comes in the 1st possibility. If it is the case we have to insert the patient record(PatientA) to the same group id, in this scenario group_id is 9. Not only that all other patient records which exhibited a matching property with that patient PatientA should come to the same group id 9 under the report_id 48.

This is the code that I wrote for this task.


More details about the changes can be found here.

Within this week I also thought to lay the foundation for the next task as well. A ticket has been created for the the task by name PTM-84. The main target of this task is to perform the match with all the records. In the current version it only allows to perform match with the same set of records by the property of deduplication. 

Sunday, June 18, 2017

Week 3: Finally Some Relief

I am very happy to say that the problem that I have aforementioned in my 2nd week's blog post, has already been solved. 

In my 1st week's blog post I did mention the tasks that I should complete withinin this summer. The task I have completed in this week is,
Once the user has selected the old report, we must find the date it had run and fetch the patients' records which are either added or updated after that date. This is done by comparing patientmatching_report.created_on AND patient.date_added OR patient.date_changed table fields.
 This is how it was mentioned in the post.

There were two classes that I should change to get the job done,
Changes related to those classes can be found here and here respectively.


Things I want to highlight


According to my target I should maintain a single report per a strategy therefore it is necessary to have unique name for a strategy. For example a strategy having the name "family_name block" will be named as as "dedup-incremental-report-family_name block". 

How did I Fetch the Patient Records ?
It is straightforward. As I have to fetch the patients' records which are either added or updated after the date of the report generated, I had to add a restriction to the criteria which was created by hibernate createCriteria(Patient.class).


Only for a Single Strategy
In order to achieve the above part it must be ensured that this only happens when the user selects a single strategy. This part was done considering the user's selection. If there is only a single strategy the above part will get activated. I have shown in the below, how the code appears to be in the class MatchingReportUtils.java inside the method InitScratchTable


Sunday, June 11, 2017

Week 2: Struggling Times

Since the 2nd week has almost finished, my primary goal is to get the incremental patient matching into alive, as aforesaid in my 1st week’s report. I have split it into sub goals in order to get a clear picture on it. 

Here are the sub goals,
  1. The strategy should be identified in which the user is being used.
  2. Is the strategy a combination of set of strategies? If it is so, we ignore it. Otherwise we should consider the particular strategy and then should follow the steps as they are listed below.
  3. Should find out whether there are any reports stored in the database related to the strategy. If there are no reports, we are good to go. Nothing has matched earlier therefore all the patient records should be matched with each other.
  4. Once the user has selected the old report, we must find the date it had run and fetch the patients' records which are either added or updated after that date. This is done by comparing patientmatching_report.created_on AND patient.date_added OR patient.date_changed table fields.
 My primary work consists of 1,2 subgoals and it is almost done. I have started creating the foundation for the sub goal 3. Since I was curious and interested about working with patients I paid my attention to the 4th goal at the beginning of the second week. Believe me, I have been struggling for the past 5 days just to find the code segment which is meant for retrieval of patients from the database. It’s just a matter of time of finding the code segment because I have already found a way to retrieve the patient records in order to adhere with incremental matching process.

The following two tables are the data sources which are meant to help in completing this task, 

1. patient

2. patientmatching_report

By comparing the patient's table date_changed or date_created with the field created_on in patientmatching_report I can simply sort out the necessary records for the next match.

I have figured out the place where the patients are retrieved,

This is the method which loads all the patients regardless of the date created or date changed.


This week was a tough week for me as my end semester exams have already started, somehow I managed to allocate time for this wonderful Patient Matching module. 
My next goal is to complete the 4th sub goal.







Monday, June 5, 2017

Week 1: The Match Begins!

The objectives of the Patient Matching 2.0 project are,

1.  Perform patient match taking considerations of the previous run (incremental patient matching). 
  • This method will avoid unnecessary record comparisons.
  • A match will be performed upon,
          addition of new patients or
          changes in patients’ records
  • After an incremental match a separate output will be created.
  • This incremental matching is specific to a strategy.
  • An output from previous run can be selected as the memory for the next run(eg. For strategy “A” there are 50 runs in last 3 months, so it should be possible to pick one output from the 50 as the memory for the next match)
2.  Avoid repeated manual reviews of previous manually reviewed matches
3.  UI changes to reflect above functionalities

According to my project plan my first goal is to perform patient match taking considerations of the previous run (incremental patient matching).
I have already identified a method to get the work done. For my first goal the relevant tables in OpenMRS database are,

  • patient - has all the information about when was the patient added and when was the patient last updated
  • patientmatching_configuration - has the details of the saved configurations
  • patientmatching_report - has the stored strategies
  • patientmatching_report_configuration - associate table which match the table patientmatching_configuration and the patientmatching_report
  • patientmatching_report_generation_step - has the properties which should be updated as in the  incremental matching process report is going to be updated without creating some bunch of reports over and over again.
Below images are the screenshots of above mentioned tables.

patient


patientmatching_configuration


patientmatching_report


patientmatching_report_configuration


patientmatching_report_generation_step



I have identified few steps to solve this problem. 
In the code, 
  1. The strategy should be identified which the user being used.
  2. Is the strategy combination of set of strategies? If it is we ignore it. If not we should consider the particular strategy and then should follow the steps listed below.
  3. Should find out whether there are any reports stored in the database related to the strategy. If there are no reports, good to go nothing has matched earlier therefore all the patient records should be matched with each other.
  4. If there are couple of old reports related to the strategy, should be prompt to user asking which report is considered as the memory for the next run.
  5. Once after the user has selected the old report we must find the date it has been run and fetch the patients' records which are either added or updated after that date. This is done by comparing patientmatching_report.created_on AND patient.date_added OR patient.date_changed table fields.
So what I have been currently doing is the 1,2, and 3rd steps. My GitHub repo can be found here.