Amazon ReMARS 2019 Conference: Personal Impressions
1. Executive Summary
I have had the fortune to attend to the Amazon ReMARS 2019 conference focused on cutting edge AI/ML, Robotics, IoT and Automation technologies. I have tried to attend as many sessions as I could from Tuesday through Thursday. You will find a summary of my observations of those sessions in the following sections.
There were several keynote speeches some of which were quite entertaining. The main message was that Machine Learning (ML) technology is now mainstream and very accessible to everyday developers thanks to the availability of open source libraries that package cutting edge algorithms and products such as Amazon SageMaker that makes it very easy to consume such libraries to build applications that can be easily deployed to the production hardware on Amazon cloud.
There were lots of emphasis on the automation and robotics and how there have been significant developments in the robot vision, dexterity (handling) and autonomy/movement. Such developments are obviously happened on the back of recent developments seen in the ML technology.
Although there is still a lot of hype around some of those technologies, I have seen that with appropriate focus and sound understanding of the real underlying technologies, truly innovative solutions could be provided. This is what the team at the Australian energy company Woodside did: they married the good old system and software engineering with emerging technologies such as IoT, ML, computer vision and the cloud computing to build a completely automated, intelligent remote natural gas site that is very easy to operate.
2. Tuesday Workshop Sessions
2.1. W07: Voice Control for Any Thing
This session was about using an Arduino IoT kit to build a simple IoT device which consisted of a set of flashing LED lights arranged on a breadboard and connected and controlled through a Arduino-based micro computer running an embedded Linux operating system. The microcomputer had all the basic interfaces you would expect from a modern computer: USB ports, HDMI port, Ethernet port to connect to internet and usual PC monitor and keyboard.
As interesting as it was to have a micro computer of the size of your palm running a complete Linux system with live connection to the Internet, the real excitement was to control the LED IoT device through the internet connection of the microcontroller from the AWS cloud through a set of AWS services. Such services could be integrated to the Amazon Alexa, hence augmenting Alexa’s capabilities with custom hardware in the form of an IoT device.
This was a very hands-on session and all that breadboard, connections and devices reminded me of my electrical engineering days!
2.2. W05: Finding Martians with AWS RoboMaker and the JPL Open Source Rover
This session was about using the open source robot integration framework RoboMaker to build a simple rover application on AWS and to simulate and control the rover through the tools that are available through AWS RoboMaker product.
We learned the basic concepts of RoboMaker framework which is a scalable, node-based processing architecture that allows robot developers to build sophisticated robotics systems from a set of building-block components (e.g. a camera node to provide vision, a radar node to handle radio signals, etc.) and connect them through a message processing architecture (similar to how Windows operating systems works with handling window messages).
AWS RoboMaker makes creating, building, deploying, running (for simulation or processing) and scaling robotic applications as easy as possible. I likened the whole process to wizards you see in development IDEs like Visual Studio. You specify what kind of stack you want and AWS RoboMaker generates the whole software project on the cloud for you and it deals with all the hardware infrastructure you will need to run the code on the fly (IP network, compute and storage nodes, service entry points, etc.). It was almost too easy! As a developer you just focus on the core logic of your robot application and let AWS’s framework to handle the rest. I quite liked the concept and seems like a very useful product to quickly prototype robot-based applications.
3. Wednesday Sessions
3.1. M10 - Infor Coleman: Streamlining Enterprise ML Complexities
This session was about how data analytics and Machine Learning technology being used by one of the major ERP vendors in mid-market segment, Infor Coleman, to enhance the functionality of the traditional ERP applications: inventory management, accounting, production planning, sales and distribution, etc. The benefit of the use of visualization backed data analytics is obvious in such applications. What was interesting was their use of ML techniques to augment/enhance their MRP (Material Resource Planning) algorithms to make them more dynamic and flexible with respect to the changing market and operating conditions thanks to making them data-driven.
One of the major challenges mentioned by the speaker when it came to the use of advanced ML technology like deep learning was the model explainability. In tightly controlled environments such as where usual ERP software is deployed, you cannot just act based on the signals from a black-box. Decisions have to be possible to be traced back to the logic/drivers that generated them. This is the case with basic ML models like decision trees, random forests etc. but not the case with the ones based on neural nets even though the latter seem to generate better results overall. So there is still more work to be done in that space for production deployment of such models.
3.2. M20 - From Business Intelligence to Artificial Intelligence
This session was about the journey of a large chemical company, INVISTA, in building and operating a data analytics platform. Like any large organization, they were familiar with data analytics through data warehousing and business intelligence applications. They wanted to move from that to a more integrated and automated architecture where more advanced data analytics could be delivered in a more-or-less predictable way.
One of the most important lessons they learned was to form mixed-skill teams (e.g. a data scientist, one or two domain experts, few programmers, etc. all part of the same team) instead of forming teams along skillset lines (e.g. data scientists, programmers, domain experts, etc.). This allowed them to do agile style quick iteration cycles and develop their platform quickly and organically without losing the trust and confidence of their users.
3.3. R11 - The Open-source Bionic Leg: Constructing and Controlling a Prosthesis Driven by Artificial Intelligence
This session demonstrated a real-life bionic leg which was built by an academic team initially through basic finite state-based model of a leg. When you think about it, a leg could be in certain number of states: standing, walking, climbing up or down, etc. These states are important because they determine the sort of forces and torque the leg hardware has to exert to allow safe and stable operations. In fact, we were told that the correct state determination and smooth state transition was the most challenging part of building a robotic leg for humans.
The team developed a simple Finite State Machine (FSM) model of the leg first. Later on, they used the data coming from the sensors to train Convolutional Neural Network (CNN) based models to fine tune this FSM model to make state transitions smoother. The interesting thing was they did not give up their original FSM model because that model is simple enough to understand and to visualize the operation of the leg to get a good intuition behind the more complex CNN models.
3.4. A29 - Sooner Than You Think: Neural Interfaces are Finally Here
I was not quite sure about what I would learn in this session as I always found the idea of tapping into the brain signals as a hopelessly futile sci-fi fantasy. I was pleased to find out the presenters of this session were of the same opinion and they approached the problem of finding out what is going on in the brain from a completely different and novel angle. Instead of trying to understand and reverse engineer something as complex as brain by looking at brain electromagnetic activity — as many other researchers seem to be doing — they accepted the brain as a black-box system with very well-known and understood IO ports: nerves that connect brain to the muscles through the spinal cord. Such nerves enable motor actions/reactions in the muscles and it is much more easier to both measure and pinpoint/separate specific signals coming from the brain.
In the demo, they have shown how a user can play a simple computer game by just moving his fingers around. There was no computer vision etc. involved. The user would wear a device which measures the electromagnetic signals detected on the skin resulting from muscle motor actions and those signals could be mapped to the certain parts of the hand (e.g. middle finger). The device will send that signal to the receptor (computer in this case) through a radio signal and that way it could be actioned upon.
I found this technology quite exciting and think that it could be even used to detect the speech signals (if you place some sensors on the tongue and throat to detect muscle movements associated with speech synthesis process)
3.5. R08 - Creating the Intelligent Asset: Fusing IoT, Robotics, and Artificial Intelligence
This was one of the most interesting and enjoyable sessions I attended to at the conference.
The presenter was from Australian energy company Woodside and he demonstrated a very concrete use of technologies that were subject of the conference in a specific case of remote gas plant management operations.
Woodside’s problem was to manage a remote natural gas plant remotely with minimal(i.e. no on-site) human involvement in an effective and cost efficient way. They deployed various IoT devices around the plant --some of which they had to design and develop themselves for cost related reasons — to take measurements and to send/receive signals so that they can be remotely controlled. This far it is standard SCADA we are already familiar with. What they did innovatively was to create a 3D model of the entire plant and build an software control application that allowed operators to walk through this 3D model (like in Google earth/street view) and be able to see and operate each and every part of the plant remotely!
Each part of the CAD model they built is reactive to the user interaction through the software (e.g. click to see most recent measurements etc.). Moreover, they have integrated a rich data analytics platform on the parts of this model where an operator not only can see the current measurements but also the history of them together with trends in the measurements, associated graphs and all related data in the same application view. It almost made the job of a plant operator as that of someone playing the Sim City. It seemed like an amazing application.
I asked few questions how they developed the whole system and it turns out that they used a cross functional team of 10-12 people (developers, data scientists, domain experts, etc.) over a period of 2 years.
4. Thursday Sessions
4.1. M39 - Scaling Photo-realistic 3D Content Creation with Machine Learning
This was another very interesting session. The presenter is the development manager of Amazon imaging technology who are tasked with generating 3D CAD images of the products you see on the Amazon.com. 3D models of the product products have lots of advantages over the photos: they can be looked at from different angles/perspectives, they can be “teleported” into virtual settings to let customer know how they will look in different contexts (e.g. sofa in a living room).
The labor intensive way of 3D model generation is to have 3D artists and let them create the models. This is not only very expensive but — more crucially — does not scale in Amazon’s product range scale. They simply do not have that many 3D artists to create the models for every one of the millions of the products they sell. The solution they have in mind is to find a way to automatically generate the 3D models from product descriptions.
I have to say the most of the session was about the problem definition rather than the actual solution. As far the solution is concerned, the presenter stated that he believed the solution lies along the spectrum of machine vision and machine learning models. He did not get into the details at all, perhaps it is still unsolved problem and/or it is Amazon’s R&D secret.
I asked him about whether structural, ontology-driven product description could be used as well and he gave affirmative answer as they are looking into that approach as well.
So, in the absence of Amazon’s actual solution, I took the liberty to come up with my arm chair engineering design for the benefit of the reader:
-
Have a structural description of products and parts. For example, have a concept of chair which consists of legs and a base and optionally a back stand. This part-whole relationship is important to compose complex CAD models from constituent parts.
-
Use machine vision algorithms to recognize objects from their photos/pictures and map them to the items in the product inventory. Use the ontological whole/part description of the product to further recognize individual parts of the product.
-
Build a repository of CAD models for some of the parts of products we come across. This will be a labor intensive exercise.
-
Use Machine Learning models to map the images of the part of a new product to a CAD model. This ML model will be trained using the data in the repository described in the third step above. There are some open research problems of generating highly structured mesh data — that is characteristics of 3D CAD models — as an output of a ML model.
-
Build a 3D CAD synthesis model that can build a complicated model from simpler parts just like you build a structure from LEGO parts. This should be straightforward. This process will use the structural/ontological description of the product mentioned in the first step above.
4.2. M30-L - Investing in Technology Breakthroughs
This was a fun session by a big shot venture capitalist who was talking fast enough that if you had momentarily lost your interest you would miss several complete sentences. Fortunately what he was talking about was very attention grabbing laced with jokes and good humor I believe I did not miss much.
One of main take away for me was how advanced, innovative technologies are linked to each other. He was stating the widely known fact that no one can for sure will know which technology will be successful and which startup will prevail over a 10 year investment horizon of a typical VC investment cycle. However, by virtue of investing in some of those companies and taking interest in their products and technologies you can find hidden opportunities and links allowing jumping to the other areas/products/technologies you never even thought about. This is bit like some of the most successful startups like Slack starting as companies with completely different goals and finding their successful products (Messaging app for Slack) almost by accident. His point was by investing in cutting edge companies and taking an interest in their technologies and connecting "the dots", one can significantly increase the chances of coming across these "accidents".
4.3. A03-L - Follow the Money: What We Can Learn from Venture Capital Investment in Automation
This was a session presented by a veteran of American Robotics Association who has been in the robotics industry since 70s and have witnessed many hype and disillusionment cycles over the years.
One of his interesting messages was that the increase in robotic automation in US is positively correlated with the employment (in other words, negatively correlated with unemployment). He was stating this empirical fact to claim that some of the worries around automation taking away jobs were overblown. Personally, I am not sure how that correlation will hold over the next 40 years.
Another of his main talking points was the observation that even though high profile robotics products/applications such as Boston Dynamics robots take the headlines as if these sort of developments suddenly took place, the progress in the industrial robotics has been steady since 70s with gradual improvements. There is certainly some acceleration thanks to the use of big data, advanced ML models and cheap and powerful computing power to endow robots with vision, dexterity and autonomy.
4.4. A22 - A Hype-free and Cutting-edge Discussion on Autonomous Driving
This was a historical overview of the development of self-driving car technology from one of his veterans who got involved in the field from mid 90s through the academia. One of most striking observations the presenter made was they had the self-driving car demo — the sort who gets people excited nowadays — back in late 90s, perhaps less presentable with big sensors sticking out but functionally almost equivalent.
Improvements in the sensor technology and the increase in the compute power through GPUs have been the two major technical enablers of the self-driving car technology. The main challenge of safety remains the same, especially in dense urban areas where self-driving cars depend on the availability and accuracy (i.e. maintenance) of high definition maps to allow vehicles to have precise control at the accuracy of 1/1.5 meters.
One of the most interesting insights from this talk was that it is most likely to find self-driving vehicle technology to be deployed first in hybrid settings and in small physical scales rather than in fully automated large scale self-driving scenarios:
-
An example for the hybrid setting would be self-driving trucks being autonomous on the highways — where the driving is more predictable and hence safe and GPS and camera/LIDAR guidance is sufficient — but being driven by humans in and out of city centers to truck ports outside city boundaries.
-
An example for the small physical scale vehicles would be small carrier/console like vehicles (e.g. grocery shop carrier) which would pose no harm to humans because of their the low operating speed and weight.
4.5. R14-L - Every Tool’s A Hammer: Life is What You Make It
I was expecting this to be a maker’s how to session but it turned out to be more of social/philosophical/leadership session where the speaker — a famous TV presenter active in makers show scene — shared his philosophy about how to design things, get them built and share them with the community.
It was full of leadership and life lessons and I did not regret that he did not talk about his latest gizmos or maker projects. One of the interesting things he talked about was how he learned to delegate tasks and let others have a go at the some of the work that needs to be a done in a big project. Another person is very unlikely to do the things exactly how you would do but if you have the right perspective about this, you can view this as an opportunity to perhaps learn something new or look at a problem from a completely different angle, both of which could be very rewarding for the receptive minds.
No comments:
Post a Comment