Archi & Techno

Archi & Techno

Visualizing massive data streams: a public transport use case

www.octo.ch

Public transport companies release more data every day and some of them are even opening their information system up to real time streaming (Swiss transport, TPG in Geneva, RATP in Paris are a couple of local ones). Vast lands are unveiled for technical experimentations!

Beside real time data, these companies also publish their full schedules. In Switzerland, it describes trains, buses, tramways, boats and even gondolas.

In this post, we propose to walk through an application built to visualize, in fast motion, one day of activity, as shown in this movie. As real time data are not yet available, they were simulated, based on available schedule information. This pretext is too good not to dig into a stack containing Play/Scala/Akka on the backend, Angular2/Pixi.js/D3.js/topojson in the browser, linked together by Server Side Events.

This prototype is intended to explore the possibility of doing massive geographical visualization in the browser, applying techniques described in a previous post.

The backend and frontend code is available on github, and tests continuously ran on travis-ci.

Read more

Archi & Techno

How does it work? Kubernetes: Episode 4 – How to Ansible your CoreOS, and etc(d)!

Hey everybody,

TL;DR

I hacked something together in order to create a Kubernetes cluster on CoreOS (or Container Linux) using Vagrant and Ansible.

If you keep reading, I’m going to talk to you about Kubernetes, etcd, CoreOS, flannel, Calico, Infrastructure as Code and Ansible testing strategies. It’s gonna be super fun.

The whole subject was way too long for a single article. Therefore, I’ve divided it into 5 parts. This is episode 4, regarding the usage of Ansible on CoreOS, and etcd.

Read more

Archi & Techno

How does it work? Kubernetes: Episode 3 – Infrastructure as code: the tools of the trade

Hey everybody,

TL;DR

I hacked something together in order to create a Kubernetes cluster on CoreOS (or Container Linux) using Vagrant and Ansible.

If you keep reading, I’m going to talk to you about Kubernetes, etcd, CoreOS, flannel, Calico, Infrastructure as Code and Ansible testing strategies. It’s gonna be super fun.

The whole subject was way too long for a single article. Therefore, I’ve divided it into 5 parts. This is episode 3, regarding Infrastructure as Code, and the tools of the trade.

Read more

Archi & Techno

How does it work? Kubernetes: Episode 2 – Kubernetes networking

Hey everybody,

TL;DR

I hacked something together in order to create a Kubernetes cluster on CoreOS (or Container Linux) using Vagrant and Ansible.

If you keep reading, I’m going to talk to you about Kubernetes, etcd, CoreOS, flannel, Calico, Infrastructure as Code and Ansible testing strategies. It’s gonna be super fun.

The whole subject was way too long for a single article. Therefore, I’ve divided it into 5 parts. This is episode 2, regarding Kubernetes networking.

Read more

Archi & Techno

How does it work? Kubernetes: Episode 1 – Kubernetes general architecture

Hey everybody,

TL;DR

I hacked something together in order to create a Kubernetes cluster on CoreOS (or Container Linux) using Vagrant and Ansible.

If you keep reading, I’m going to talk to you about Kubernetes, etcd, CoreOS, flannel, Calico, Infrastructure as Code and Ansible testing strategies. It’s gonna be super fun.

The whole subject was way too long for a single article. Therefore, I’ve divided it into 5 parts. This is episode 1, regarding the Kubernetes general architecture.

If you want to try it:

git clone https://github.com/sebiwi/kubernetes-coreos

cd kubernetes-coreos

make up

This will spin up 4 VMs: an etcd node, a Kubernetes Master node, and two Kubernetes Worker nodes.  You can modify the size of the cluster by hacking on the Vagrantfile and the Ansible inventory.

You will need Ansible 2.2, Vagrant, Virtualbox and kubectl. You will also need molecule and docker-py, if you want to run the tests.

Read more

Archi & Techno

An IoT application using IoT framework? Here it is

According to the market researchers at IDC, there were 9.1 billions Internet of Things (IoT) devices installed at the end of 2013. They expect that number will increase stably and will reach 28.1 billions in 2020. In front of this huge number of connected devices, there is an enormous potential market of thousand services and application products, which will innovate rapidly at the same time just like the explosion of Android or iOS applications.

How could the relevant company adapt to this market environment? More accurately, how could the developers in this kind of company create a quality application more efficiently? The answer may be using a powerful framework to simplify the development and reduce the development time.

In this article, we will give a brief introduction about Standard OneM2M and its implementation OM2M. In order to give a more concrete global view of OneM2M as well as OM2M, we realized a PoC dedicated to a connected vehicle user case, in which we implemented two classic services (GPS tracking, real-time speed display and remote control).

Read more

Archi & Techno

“pet vs. cattle”, from server craftsman to software craftsman

The evolution of Ops follows a path that we observe regularly in our interventions. It is through this fable that we will see the 4 stages that mark this path paved with pitfalls. Let’s see how an Ops proceeds concretely to carry out the operation “fix_mysql” which consists in changing the configuration of MySQL on production servers.
cw_mfpdxcaugw_k
Read more

Archi & Techno

D3.js transitions killed my CPU! A d3.js & pixi.js comparison

www.octo.chD3.js certainly is the most versatile JavaScript data rendering library available: turning data into mind blowing visualizations is only limited by your imagination. A key component to turn static pages into animated ones are the powerful selection transitions. However, too many simultaneous transitions on a web page will soon bring you CPU on its knees.
Hence this blog post.

We faced this problem when displaying swiss transport real time data on a map, within an SVG layout: rendering was lagging, event sourced data were not consumed consistently and laptop batteries were drowning at a dramatic speed. A video from a first attempt can be seen, and compared to a newer implementation with the technique presented in this article. Another surprise came from rendering a simple clock, burning 20% of CPU with a single transition.

If d3.js has no serious concurrents for many rendering problems, we decided to give try to a JavaScript library used for building games and leveraging the strengths of HTML5 and GPU: pixi.js.

At first, we will propose in this post a comparison between the two libraries in terms of rendering performance. For the sake of completeness, we will also discuss native CSS transitions. We will then dive into a couple of tricks to enhance dynamic visualizations with each of the two libraries and will even combine them to get the best of both worlds.

The project source code with benchmark data are hosted on github and a demo is available on github.io.

Read more

Archi & Techno

Android Styles & Themes for developers

For beginner Android developer or the more experienced ones who don’t do much of the UI work, understanding the difference between styles and themes and how they should be used can be very difficult to understand.

With AppCompat being a must have in every app and as it’s relying A LOT on themes and style, understanding all this can be very frustrating when it comes to customizing its default behaviors.

With this article, I’ll try to explain what this is all about and how it can helps you into your app’s UI code.

Read more

Archi & Techno

Joyful wind of change: A software craftsmanship short tale

This is the story of a team. A bunch of 11 aspiring software craftsmen who decided to change things around and get their job done in a better way.
The story takes place between the 30th and the 50th iteration of the development process of a software. This software is a website serving over 2 million regular users and providing legal information and services to 65 million French citizens.

Chapter one:
Start from what hurts and set a direction

Leaky pipeline

It is normal that the build fails. There are some automated E2E tests that fail randomly. Just try again, hopefully it will work next time.

This was the common answer to all new developers who struggled passing their first user story. And for information, the build then used to be about 45 minutes long. Almost nobody would get shocked anymore, as if everyone just got used to the pain. Read more