Hadoop in my IT department: benchmark your cluster
The stress test is a very important step when you go live.
Good stress tests help us to:
- ensure that the software meets its performances requirements
- ensure that the service will deliver a fast response time even under a heavy load
- get to now the scalability limits which in turn is useful to plan the next steps of the development
Hadoop is not a web application, a database or a webservice. You don’t stress test a Hadoop job with a heavy load. Instead, you need to becnhmark the cluster which means assessing its performances by running a variety of jobs each focused on a specific field (indexing, querying, predictive statistics, machine learning, …).
Intel has released HiBench, a tool dedicated to run such benchmarks. In this article, we will talk about this tool.



