This article explores some key aspects of data lineage at the data set level, indicating how to hack the Spark engine to achieve this.
Read moreYou're running late, and not just a little late. You throw your bag under the table and boot up your PC. “Hi! Sorry I’m so late! You say hello to Jeremy, Audrey, and Farid. Audrey says: “Maria came by. She was looking for you.” “Did she say what for?” Hmmm - ‘I’m waiting for an action plan. I don't see anything coming. I'm worried’ “OK” Not OK. This action plan that Maria is waiting for has been spinning in your head for over a week…
Read more“Kubernetes is the Linux of the cloud”This quote by Kelsey Hightower during the Kubecon 2017 in Austin emphasize the rise of Kubernetes among modern cloud infrastructures. This rise is partly driven by the developers community, but also by the web giants such as Google, Amazon, Alibaba or Red Hat who have invested a lot on this technology, and keep contributing to its improvement and smoothening its integration to their respective ecosystems. EKS for AWS, GKE for Google and AKS for Azure are good illustrations of…
Read moreThe Eye-Brain Law: To a certain extent, mental power can compensate for observational weakness. The Brain-Eye Law: To a certain extent, observational power can compensate for mental weakness. Jerry Weinberg "I received your email. I get what you want to do. But, what's the goal?" Comfortably secured in a state-of-the-art ergonomic seat, and protected by a natural fortification of edible, combustible, destructible, fungible, flexible, securely-fixed, decay-prone, sensitive and transmissible elements, Mazare observes both you and Jeremy. Jeremy answers, "We would like to analyze the lead…
Read moreWhile the interest for the Internet of Things by companies is no longer to be proven, this area continues to give a hard time to experts, as for security and architecture. Indeed, the multiplication of data sources brings a reflection on the architecture of networks. As Satya Nadella (Microsoft CEO) said on stage in 2017: "When I joined Microsoft in 1992, all Internet traffic was 100 gigabytes a day. Today is 17.5 million times that amount ... per second! "And we are only at the…
Read moreSince practical computation demands that implicit assumptions be brought out into the open, it is no coincidence that computer programmers are attracted to an approach devoted to studying how people make assumptions. -- Gerald Weinberg. number: 4240 date: 08/29/2017 status: in progress type: bug severity: serious submitter: C. COURDEL nature: partial budget carry forward does not work description: I skip the budgeted year, perform a partial carryover, option: without exceeding, I submit the form, nothing happens! Owner : _ Resolution status: _ You've just opened…
Read moreThe treatment of error as a source of valuable information is precisely what distinguishes the feedback (error-controlled) system from its less capable predecessors. Jerry Weinberg @OlegTxl Direct Message Hi Oleg, can you spare me an hour of your time? ok, around 6pm? ping me (...) ping pong Thanks Oleg. I would like some advice about mob programming for my project at work OK I was given three months to turn the situation around on the app I'm working on situation ==? we have to deliver…
Read moreA few months ago our friends Guillaume Lefevre and Etienne Coutaud made an appearance at the 2017 KubeCon to present their work on a monitoring solution based on Prometheus and Grafana. (The video can be found here) A bit over a month later, we faced a problem in production, the platform started to behave abnormally. The graphs showed bursts in metrics which was unusual and some people were wondering if a deployment could have changed something in the configuration and triggered the problem. Even if…
Read moreManagers are not confronted with problems that are independent of each other, but with dynamic situations that consist of complex systems of changing problems that interact with each other. I call such situations messes. Problems are extracted from messes by analysis. Managers do not solve problems, they manage messes. Russell Ackoff A chat window opens at the bottom right. Maria: come to see me when you can for 15 minutes today pls. You save your work, answering: I'm coming. You grab something to write with.…
Read moreHi all, Happy new year! 2017 is now over, and it’s time for us to contemplate everything that happened in Octo Technology Australia and appraise our success and failures. Hidden Tech Challenge Late 2017, we launched a geolocated quiz game in Sydney, and it was a lot of fun. Not only did we build the product from the idea to the delivery, but we also handled the communication, animation and follow-up of the game.
Read more
