reading huge xml in ruby

Dealing with huge xml? say of size 400 MB or more than a GB. No worries, Nokogiri::XML::Reader is there for your help. But let me tell you it takes a while before you understand it and start using it. Needs to understand types of elements, and when to read value and and when to read attributes. However I have went over it and tried defining my own module which now simplifies it.

Your large xml doc must be having thousands of records, and you may or may not know the structure of record. This module will help you easily read your records, its elements, something like this below …

hey_my_module
  take_this_xml
  and look_for_these_elements
  let_me_know_if_you_find_any_of_them

…hence

HugeXML.read xml, elements_lookup do |element|
  # => element{ :name, :value, :attributes}
end

so simple right? wanna check it out… here it is  => github.com/amolpujari/reading-huge-xml

 

About these ads

2 thoughts on “reading huge xml in ruby

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s