If Athena only evokes this (traumatic, isn’t it?) scene to you, you’ll be disappointed: this blog post is dedicated to Amazon Athena, the latest analytic tool recently announced at Re:invent 2016.
This is the story of a team. A bunch of 11 aspiring software craftsmen who decided to change things around and get their job done in a better way.
The story takes place between the 30th and the 50th iteration of the development process of a software. This software is a website serving over 2 million regular users and providing legal information and services to 65 million French citizens.
It is normal that the build fails. There are some automated E2E tests that fail randomly. Just try again, hopefully it will work next time.
This was the common answer to all new developers who struggled passing their first user story. And for information, the build then used to be about 45 minutes long. Almost nobody would get shocked anymore, as if everyone just got used to the pain. Read more
My first project right after school was called KISS which means
Three years and a few IT projects later, I thought that I got it right and applied the following rules:
Yet a few weeks ago, I learned that you can always remove useless things. You just don’t know it yet.
I wanted to create an API above the drugs data set (freely provided by the French government as CSV files). I knew that I needed a script to parse and store the data and a web API to expose them. The code is on github and the application is live.
When I start a web application, I have 2 main choices to make:
After a while I needed “full text” search, so I added elasticsearch, simply because I knew how to use it. It requires a few (ugly) scripts to automate its synchronization with couchbase.
And suddenly came the epiphany! Or to be more precise, I asked for a review and after detailing the context, my colleague simply asked: “why don’t you remove couchbase?”. He was spot on: the dataset fits in memory and it is readonly.
It was replaced with a simple import to load the JSON in memory ready to be queried by the application.
But it got me thinking, couldn’t I remove the second database too? And I learned that there are indeed multiple libraries to search into local documents: I just used lunr and could therefore remove both databases.
The application is now more simple and reliable:
Nonetheless, after the refactor, the application was a bit slower even if everything was done in memory. Having said that, response time is still below 20 milliseconds, so the advantages brought by this removal still exceed the loss of a few milliseconds.
This resulted in a pull request with twice as many deleted lines as added lines and my takeaway would be:
Following KISS principles is more difficult than it seems: you can always question yourself but you should more likely seek for feedback from your peers. And, instead of choosing a database straight away, I would now choose a place to store my data.
In 2012, Facebook teams were in the process of rebuilding their native mobile applications and they had to adapt their server queries to get the data they needed. When they started to refactor some parts of their code, some got frustrated and stepped back to think of a better way. GraphQL was born.
They changed their point of view in terms of resources, and they preferred the idea of a graph of objects instead. The whole idea is to call a single endpoint, passing a query as the content of the request.
Throughout this post, I will give an overview about what GraphQL is, including advantages and drawbacks. Then will come the best part with being how to create a GraphQL API.
In this post, we will make a feedback on a surprising bug we had to solve due to the auto-magical changes detection of Angular 2.
The blockchain is a trendy subject, and we believe one needs to apply its principle to an actual project to understand its technology and progress status. We chose to implement an online money pot in Ethereum, a kind of blockchain that focuses on smart-contracts instead of money transfer. This article demonstrates how we can leverage Ethereum to develop this use-case.
by Alexandre Masselot (OCTO Technology Switzerland), Catherine Zwahlen (OCTO Technology Switzerland) and Jonathan Gianfreda.
The possibility of custom plugins is a strong Kibana promise. We propose an end to end tutorial to write such plugins. But this “end to end” approach also means “how to continuously deploy them?”, “how to share an environment with seeded data?” Those questions will bring us in a full fledged integration infrastructure, backed by Docker.
The Elasticsearch has grown from a Lucene evolution to a full fledged distributed document store, with powerful storage, search and aggregation capabilities. Kibana has definitely brought a strong component for interactive searching and visualization, transforming the data storage tier into an end user browser.
Customizable dashboards via a rich library of graphical components made its success, but soon, the need for real customization arose. If plugins were thought to be integrated from early on, the actual customization often lied into forking the master project and adapting to on particular purpose. Merging back fixes was soon to be a daunting effort to keep up with the high pace of the Github repository evolution .
So, that should be easy. Just google and you would craft wonderful shiny visualizations.
But not fast, young Kibana Padavan! Documentation lacks, resources are valuable but scarce. But the promise is still shiny and we want to reach it.
In this post, we propose to share our journey into the writing of Kibana plugins, the little pitfalls we fell in and the setup of continuous deployment into a Docker environment. There is no dramatic discovery or stunning breakthrough today, but a tentative to write a map to make your journey easier.
A smooth user experience is a common attribute of all the successful Android applications. It might sound as an obvious statement but many are the applications that are a little bit “laggy”. Moreover android developers are use to test their application mostly on high-end devices (generally their own devices), forgetting that their app will be run on cheaper devices, which will deteriorate the app smoothness a lot.
It is pretty easy to find documentation on the Internet that will give you advices to have an efficient android application (for instance the great android performance documentation page itself). In this article we propose the opposite: presenting tools that will help you benchmarking an existing Android application in order to find some room for improvement. It can also be a way to measure improvements when we are doing some performance related tasks. We will try to have first a macroscopic approach and then will go deeper and deeper in the analysis in order to find the hidden cause of a performance issue.
Some of the tools I will talk about are part of the android standard build tools, some are “external” products. All of them have proven their efficiency on the projects I have worked on, including Le Monde applications which are massively downloaded (over one million downloads) and need to be performant even on low end devices .
In the first part we will see how we can identify app launch slow downs thanks to two tools: Nimbledroid and AndroidDevMetrics. In the second part we will focus more generally on User Experience (UX) issues by using standard Android tools.
Going faster on a mobile has become essential. Putting aside the actual choice on the means of communication, the data format used weighs in quite a bit when it comes to the speed. As of Today, JSON has proven to be a standard media for APIs. Still, is this data format suitable for mobile? For instance, handling JSON in an Android environment is difficult.
Other data formats are emerging in recent years like Thrift, Avro, Message Pack or Protocol Buffers.
Protocol Buffers has the ability to have a binary format that is easily adaptable and which can be manipulated. It also has a very basic structure to write and understand and easily generate source code for several languages.
This blog featured two articles on Protocol Buffers (protobuf) in 2012. The version used was 2.4.1, and the standard was proto2.
docker run busybox /bin/echo ‘hello boys and girls!’
As humans, we like new shinny things. And working in a wannabe devops world, that means solving problems with containers, too. We have been working with them for quite a while now, and we are more than happy with the results. We like them so much, that we decided to travel more than 8000 km, to Seattle, Washington, in order to go to this year’s DockerCon.
So if you’re into containers as well, and you’re dying to hear the hot news, you’ve come to the right place. Fasten your seatbelts, and enjoy the ride!