This article is the sixth episode of our MSR Interview blog series. In previous interviews we've already featured Abraham Hindle, Georgios Gousios, Vasiliki Efstathiou, Sarah Nadi and Jürgen Cito and Gerald Schermann. This week, we’re publishing the interview with Waldemar Hummer, from IBM Research AI, T. J. Watson lab in New York.
Here are some of Waldemar Hummer’s more recent publications related to MLonCode:
- A Mixed-Method Empirical Study of Function-as-a-Service Software Development in Industrial Practice
- Cloud-based Lifecycle Management for Reliable and Trusted AI
- Dependability in a Multi-tenant Multi-framework Deep Learning as-a-Service Platform
Could you please introduce yourself?
My name is Waldemar Hummer, I currently work at IBM Research AI in the T. J. Watson lab in New York. I'm in the AI Engineering division. We're building platforms, scalable infrastructures, and tools for AI, making data scientists’ and machine learning practitioners’ lives easier. I've been a part of the team that's building the IBM's deep learning platform which is part of Watson now, Watson Machine Learning, and we're currently looking more into AI lifecycle optimization.
What brings you to MSR?
I'm actually not registered for MSR this year, I gave a talk at the workshop on Software Engineering for Cognitive Services. It was the first iteration of this workshop and it was pretty interesting, lots of high quality discussions. I think it has the potential to become a key event similar to SysML which has grown out of NIPS. It used to be a workshop at NIPS and then turned into a decent conference with fairly high submission numbers. Hopefully, a similar thing will happen to this workshop. I have been to ICSE before, it's kind of my community, although I'm more of a distributed systems person, I have also done some work in software testing, repository mining, etc.
How many times have you been to MSR/ICSE?
We had a paper last year on MSR, and I think it's my 3rd or 4th ICSE.
You mentioned that part of your job is software engineering for machine learning: there seems to be no standard for documenting datasets and models, how do you do it yourself, other than just writing papers?
We have one team in IBM, they took previous publications from AI conferences and extracted the data from images and tables, reconstructed the neural networks based on the information provided in the paper. For deep learning there is one format called ONNX, it is a neural networks exchange format that Microsoft is pushing very hard, that was one way to describe the models in the platform independent way. For datasets it's different, training data is often going through a sequence of preprocessing pipelines which should also become part of the description. The metadata should include the original source, all the transformations applied, feature engineering, cleaning, filtering. This is what we are currently working on, to have a description format and a repeatable way to express those pipelines.
What is your involvement with MSR/ICSE? Have you been presenting anything?
I was an invited speaker for "Software Engineering for Cognitive Services", I probably will be part of the organization for that workshop in the future. I am not an ICSE reviewer, I am mostly a visitor this year, I have attended some workshops, most recently around cognitive services, cloud, and testing.
What do you think about machine learning for software engineering?
This is basically the scope of the project I'm currently leading at IBM, called ModelOps. It's about the interaction between the systems and AI. In the first iteration, we are building automation and systems for AI and metadata management. Once you have the information about the artifacts in your data pipeline you can make informed decision about training your models, figuring out typical patterns that one can use. If you have a declarative specification of the pipeline you can also start learning patterns and optimizing. In a large landscape of interconnected data and continuous training loops you may be able to skip some parts in your pipeline. One of the use cases we are looking into is online learning and continuous improvement for AI. IBM has a partnership with a company called Medtronic. They measure things like blood sugar levels and make predictions about different diseases like hyperglycemia. As you collect new data you can match it with predictions and update the models, it is a continuous learning loop with lots of data management challenges.
Is anyone at IBM working on applying ML to the source code itself?
I have personally done some source code mining and analysis specifically for configuration and specification of platforms like Puppet or Chef, but it wasn't machine learning in a narrow sense. There is a lot of work done for vision, speech, text, but from the top of my head I am not aware of anything regarding source code.
What was your favorite talk of this MSR/ICSE?
I really enjoyed the part of the keynote about 5G networks, the latency going down to sub-milliseconds and its impact on manufacturing. Previously you could have robots with the logic built in, that would make the decisions locally. Now you can have a central point of control and based on this single point of connectivity you can control all the machines and push the intelligence to the central point. In this case, the machines are just the actuators connected to a central “brain”. In the workshop "Software Engineering for Cognitive Services," there was some interesting work on conversational bots for reminiscence for the elderly in nursing homes. It's an area in which you can have a big societal impact by having a bot that shows a patient an image about which they have some memories and the bot can establish a conversation with the said patient. The architecture behind the bot is very complex, you have to collect a lot of data, manage all the artifacts.
Are you planning to attend ICSE next year? What do you expect to see?
Yes, I'm planning to attend ICSE next year. I think machine learning will become bigger. We have already seen some talks about program analysis and testing that use machine learning, and I would expect there will be more of that. I think ICSE is a fairly traditional and somewhat conservative community, deep down to the fundamentals of software engineering, but I am very curious to see how the conference and the workshops are going to evolve, especially the "Software Engineering for Cognitive Services" workshop. I hope there will be more understanding of what industry problems are.
What you said is what you expect to see, what would you like to see next year?
It is a hard question, but it is great to see breakthroughs in general, so I would like to see one next year.
Learn More about MSR 2019 and MLonCode: