Vote for Your Favorite Sessions for Hadoop Summit Europe!

Seismic imaging data presents significant data management and processing challenges. Terabyte-scale high-resolution datasets are routinely summarised early in workflows due to limited computing capabilities, losing valuable signal. This loss of detail restricts flexibility for geophysicists, increases numerical uncertainty and ultimately diminishes accessible insights. To test the value of the Hadoop ecosystem in this space, we devised a typical seismic processing workflow to compute a timelapse effect function of reservoir properties and automatically optimize imaging parameters to quantify them. Seismic data comprises binary format time series and was loaded using a modified version of the Seismic Hadoop library into HDFS SequenceFiles and encoded as binary arrays. We used HBase for fast random access to geophysical reference data. The workflow was broken down into a series of typical seismic processing functions which we found mapped easily onto a set of Map and Reduce functions; the embarrassingly parallel nature of these workflows made this straightforward. We describe our architecture and functionality, and show how we migrated the original sequential algorithms to the fully automated, parallel workflows using Hortonworks Data Platform, enabling seismic data processing at-scale. We developed intuitive abstractions to enable concise and flexible creation of workflows.