Problem with load_icd_desc.plx in loading icd

arnabnaha wrote on Tuesday, October 12, 2010:

Hi…
there is somekind of problem in loading icd9 2010 data into the database. after doing all the commenting and uncommenting stuff in the code and giving the privileges in the database when the loading command is run in the cmd (c:\perl\bin\perl  c:\xampp\htdocs\openemr\contrib\util\load_icd_desc.plx), it just comes up with

Inserted 0 rows , updated 0 rows.

is there a bug???

sunsetsystems wrote on Tuesday, October 12, 2010:

It looks like the web site changed and broke it.  That’s the hazard of screen scraping.

Rod
www.sunsetsystems.com

arnabnaha wrote on Tuesday, October 12, 2010:

thanks Rod,

Now what…actually i lost all my icd datas…can you please please help me out with it…if you have the icd data in your system right now, can you send me the sql file in my email. i cant wait till the code gets fixed, as i need it very urgently…
my email id: superarnab@gmail.com

please help me…its really urgent!!!

arnabnaha wrote on Tuesday, October 12, 2010:

i received them…thanks a lot….

sunsetsystems wrote on Tuesday, October 12, 2010:

Sure, done.

Rod
www.sunsetsystems.com

blankev wrote on Tuesday, October 12, 2010:

Rod,

cloud computing, took me a while, but somehow I managed to understand what they might mean with those words.

Now Screen Scraping, please explain, I feel so stupid , I would like to stay with the crowd, but I feel like I miss something.

Pimm

sunsetsystems wrote on Tuesday, October 12, 2010:

LOL.  Screen scraping means programatically extracting data from a web site by reading and parsing its pages.  Computers are not very good at figuring out what a web page means, so assumptions must be built into the program about how the pages are structured.  If the site is changed even in a minor way, these assumptions may no longer be correct and so the program must be fixed.

Rod
www.sunsetsystems.com

cverk wrote on Wednesday, October 13, 2010:

Maybe it changed with adding the 2011 codes. I was able to download the 2011 codes fine earlier this month, as they become active on October 1.  So maybe try the 2011 instead of the 2010 files.

arnabnaha wrote on Wednesday, October 13, 2010:

Yesterday, i tried with 2011 icd datas too…it was all the same…inserted 0 rows, updated 0 rows…untill Rod helped me out by sending the sql file of icd data. i imported it in the database manually…

arnabnaha wrote on Wednesday, October 13, 2010:

can anybody explain the checkout feature within the fees? it is really handy, but suppose i have a patient having two encounters with me, i take a cash payment from the patient against the first encounter (Invoice No. 000001) and save it. now when i click checkout, it shows the invoice no.000001 but as soon i do the same thing with the 2nd encounter (this time invoice number 000002) and click checkout again i get to see the 2nd invoice only. i cant see the invoice 000001. should i see it from report->financial->collection?

what if i want to print out the invoices later? i cant do that for invoice 000001 as i am only shown the invoice 000002.

kudjo1 wrote on Friday, October 29, 2010:

Hi I am having the same problem with the icd data, " Inserted 0 rows, updated 0 codes". Can you help by sending me an icd sql file to get the codes fix. Thanks. My email kudjo1@ymail.com

whimmel wrote on Tuesday, November 16, 2010:

They added a “strong” tag in between the “li” and the “a”.  Here’s a patch that worked for me.  I used the URI class because I couldn’t get the regex to work :slight_smile:

Index: load_icd_desc.plx
===================================================================
--- load_icd_desc.plx   (revision 1335)
+++ load_icd_desc.plx   (working copy)
@@ -20,20 +20,21 @@
 use DBI;              # libdbi-perl and libdbd-mysql-perl
 use WWW::Mechanize;   # libwww-mechanize-perl
 use HTML::TokeParser; # libhtml-parser-perl
+use URI;
 # or else put them into the environment with bash commands like these
@@ -75,35 +76,38 @@
     if ($tag->[0] eq "li") {
       $tag = $parser->get_tag;
-      next unless ($tag->[0] eq "a");
-      my $nexturl = $browser->base();
-      $nexturl =~ s'/[^/]+$'/';
-      scrape($nexturl . $tag->[1]{href});
+      if ($tag->[0] eq "strong") {
+        $tag = $parser->get_tag;
+        next unless ($tag->[0] eq "a");
+        my $abs_url = new URI->new_abs($tag->[1]{href}, $url);
+        scrape($abs_url);
+      }
     }

sunsetsystems wrote on Tuesday, November 16, 2010:

Thanks for finding the cause of this!  However this simpler patch seems to work for me:

@@ -81,6 +81,7 @@
     # that must be followed.  We handle those recursively.
     if ($tag->[0] eq "li") {
       $tag = $parser->get_tag;
+      $tag = $parser->get_tag if ($tag->[0] eq "strong");
       next unless ($tag->[0] eq "a");
       my $nexturl = $browser->base();
       # $nexturl =~ s'/[^/]+$'/';

I’ll commit this… let me know if you see a problem with it.

Rod
www.sunsetsystems.com

sunsetsystems wrote on Tuesday, November 16, 2010:

Oh by the way this tells me you are working with an older version of the script:

- $nexturl =~ s'/[^/]+$'/'; 

Be sure to get the current development version.  I have checked in the fix.

Rod
www.sunsetsystems.com

whimmel wrote on Tuesday, November 16, 2010:

I’ll do that. Thanks.  perl isn’t my bag

ideaman911 wrote on Sunday, December 19, 2010:

Rod;

I cannot find any script for (I assume) the icd9 loader after 9/9/10, but you mentioned you WILL commit the fix on 11/16/10 which is more than a month after that.  When I check what I think is the development CVS I cannot even find the Util folder.  Please post the specific site location here, ok?  Thanks.  I only use this once a year, but I need it now.

Joe Holzer

sunsetsystems wrote on Sunday, December 19, 2010:

Code / Git Browse.

Rod
www.sunsetsystems.com

ideaman911 wrote on Sunday, December 19, 2010:

Rod;

If I wrote you a message “Needs Justify” you might have as much insight into what I meant as I have for “Code / Git Browse”.  Do you really think anyone else understands what you mean?  Please.  Clarity comes from adequate info for someone who is NOT you to understand.

Joe Holzer

ideaman911 wrote on Sunday, December 19, 2010:

Rod;  Never mind - I figured out what your recommended fix is and it seems to have worked for the 2011 data today.

Everybody else;  After you make the mods needed for your specific $user, $dbase and $password in the PLX file, insert the line Rod shows preceded by the “+” in his code above immediately betwen the lines in the PLX file as shown by Rod, which will be the first few lines ONLY after “We handle those recursively.”, ie you will insert only that single line and only once.  Then the file should work as of today Dec 19, 2010.  The specific line to insert is:
    $tag = $parser->get_tag if ($tag-> eq “strong”);  

Rod and Whimmel;  Thanks for finding a fix.

Joe Holzer

sunsetsystems wrote on Sunday, December 19, 2010:

Joe, you can download the current script from the git repository, no need to apply a patch manually.

Also this forum is for developers, which is why you were not spoon-fed an answer.  Please post usage-oriented questions in the Users or Help forum.  It also helps when you don’t imply that the person you’re asking did not do what they promised.  Thank you.

Rod
www.sunsetsystems.com