Image: Memory Bus by ARendle, CC license.

In-memory computing, also known as massively parallel computing, is composed of two things: 1) huge amounts of RAM; and 2) huge amounts of processing power.

In-memory computing is another technology leapfrogging the traditional data warehouse. An in-memory architecture uses data that is in the main memory (also known as Random Access Memory, or RAM) of a computer, rather than data on a hard disk.

Data retrieval from a disk is the slowest part of any analytical query, because the software has to “find and fetch” the data you want, and queries accessing very large amounts of data just can’t be done in a feasible amount of time.

You’ve probably already experienced this. I work with people who launch some SAP queries that take an hour or more to run. These people would like to query even larger amounts of data but don’t even bother trying because they know SAP might just stop in midstream or take so long that the information isn’t worth the effort.

An in-memory setup eliminates “find and fetch” because the data isn’t even stored on a disk; it’s available right there in the main memory of the application, which means it is available for selection and use in your inquiry.

It also means that the way you collect, sort, analyze, chart, use and interpret data should change dramatically – from a fixed and limited process to a more natural and iterative process. The in-memory technology makes it possible to gather information in a way that is a lot like your normal thought process.

Your brain is like an in-memory computer. To make a decision, you first start with the information you have in your head. Then you gather what is missing, using the web, asking questions, reading the newspaper. Your brain immediately processes each new piece of information and sometimes in seconds you’ve made your decision.

This new paradigm – massive data storage connected to super fast computing power – will change what we ask for. No longer will we ask for a report on sales by customer, by date, by region, by product. Instead we will want every single piece of data related to any sale of anything to anyone, say, for the past two years–every single invoice, credit, return, price, discount, the person who sold it, the commission paid on it, the color of the product, the shipment date, delivery data, invoice payment amount, date of payment – everything. This will become the expectation in all areas of an enterprise.

Amazon Web Services (AWS) is one place to secure this type of environment.  The costs for 20 to 40 terabytes of storage is about the same as the monthly rent of a Manhattan apartment.