The Personal Blog of Stephen Sekula

Summer Journal: Part 2

My notes on machine learning concepts from my whiteboard at the Weizmann Institute, “dressed up” to look like blueprints.

The first half of the summer was packed with a bunch of stuff, including a busy travel schedule followed by the death of a beloved pet. The second half of the summer has been a little bit more sane, involving more focused travel followed by a break in work-related travel for some recovery before the fall semester begins. In pictures and words, below, are some scenes from the second half of the summer.


A glimpse of something beautiful

The second half of summer began with the public unveiling of something that our ATLAS Higgs research group (a collaboration of many institutions and people, including SMU) had been preparing since the beginning of the year: the use of the full 2015-2016 data sample to search for the decay of the Higgs boson into a pair of “b” quarks (“b” typically is short for “bottom,” but once upon a time these quarks were also known as “beauty” quarks). This analysis was a nail-biter. For two cycles of this search – the big Run 1 effort from 2010-2014, and the first search using just 1/3 of the 2015-2016 data – ATLAS had come up short of the standard model prediction. That’s bad if the standard model is correct, suggesting we were just unlucky and getting “downward fluctuations” of the data when we made the measurement. On the other hand, if this process is actually suppressed in nature, compared to the standard model prediction, that’s awesome – it could offer us a peek of some new principle of physics at work that act to suppress this most plentiful of Higgs decays.

When we concluded the 2015-2016 full analysis and “opened the box” to reveal the result, we found that we had seen a much more statistically significant quantity of these decays than ever before. It’s still not at the level of definitive observation – to reach that, we have to be 99.9999994% certain that what we observe is not explainable by non-Higgs processes. But right now we are just a bit more than 99.7% certain that we’ve glimpsed this process, and that is sufficient to claim “evidence” for this process. By the end of data-taking in 2018, we should have enough data in hand to know whether or not this is true, with 99.9999994% certainty (or better).

A representation of the data showing us the first glimpse of the Higgstrahlung process, with Higgs then decaying to bottom quarks. The grey represents a prominent background process, where a Z boson (not a Higgs boson) decays to a pair of bottom quarks. The red excess off to the right of the grey bump is an excess of such events that is very well-described by the Higgs boson hypothesis.

To Israel and the Weizmann Institute of Science

On July 3, I departed the US and headed for Israel, to spend almost a month working with my colleague and friend, Eilam Gross, at the Weizmann Institute of Science. He and I are in the last phase of a US-Israel Bi-National Science Foundation grant and we were looking forward to learning together about modern deep neural network and advanced machine learning techniques in preparation for a workshop at Weizmann on this subject and its applications to high-energy physics, especially Higgs physics.

Eilam and I had a very good time before the workshop, watching lecture videos on machine learning and discussing architectures and how they might be applied to problems of interest to us (my notes, kept on the whiteboard of my office, are depicted artistically at the top of this post). We barely scratched the surface of this subject before the workshop, but nonetheless the preparation was essential to understanding some of the language of the data science and machine learning communities, which (like all branches of science) have their own notation, jargon, language, and understanding of how to pose and frame problems.

Life at the Weizmann Institute in Rehovot is wonderful – perhaps a bit charmed. I quickly fell into a new life-cycle as I wiped away the 8 hours of jet lag accumulated in traveling there in the first place. I would wake around 6am and go for a run around the campus. It was too hot later in the day to run, but at 6am it was beautiful. The sun was rising by then, the temperatures typically in the high 60s-70s. At that time of day,  lots of people were already out exercising on the site. The photo above is of one of the tree-lined roads I had the pleasure of walking on every morning on my trip from my apartment on the south side of the Institute to the Physics Department and, specifically, the Schwartz/Reisman Institute for Theoretical Physics (SRitp).

Hammers and Nails

The title of the 10-day workshop on machine learning and particle physics was “Hammers and Nails.” Everybody came at this with their own interpretation of the title. The organizers thought of particle physics problems as the nails, and the hammers as machine learning tools to be brought to bear on the problems. But in my mind, I thought of the adage, “When you’re a hammer, everything looks like a nail” – a cautionary idiom that warns the listener not to acquire only one skill or tool and then try to apply it everywhere, even when it might not really apply.

Computer and Data Scientst Shai Ben-David (University of Waterloo) argues in his talk that we are only just beginning to glimpse a real “theory of clustering” – a step on the road to a theory of learning that might put modern learning algorithms on firmer footing.

It became clear to me over the course of the workshop that all is not settled in the data science and machine learning communities. They are really now only, perhaps, at the door that leads to a real “theory of learning” – or, at least, a “theory of clustering” data into like groups. In terms of understanding how it is that neural networks are so effective, there is still a great deal of open space to be explored here, and information theory and statistical mechanics are tools in tearing apart these systems to lay bare the “why” of their functionality. Generalizing beyond just neural networks, there are also people trying very hard to understand how it is that the shape of the space of input data influences the learning process. There is especially concern that the classical way of “training” a neural net – stochastic gradient descent – is reaching the limits of its applicability and may bring as much harm as good to some problems. There seemed to be a hunger for a new way to mathematically supervise the learning process.

I took away from all of this that physics and machine learning have much in common, even if at times they struggle to communicate with one another in the same language. Physicists don’t consider the universe a solved problem, and decry the description of physics as only an applied science – as some kind of “theoretical engineering.” Machine learning and data scientists similarly abhor the belief that learning algorithms are a solved problem, and decry the description of their field as purely an applied science (e.g. just make me a better self-driving car).

Day Trip to the Dead Sea and Jerusalem

On the mid-Sunday of the of workshop, we put aside slides and whiteboards and donned sunblock, tee shirts, and swimsuits and headed out for a camaraderie-filled day trip to the Dead Sea, Jerusalem, a fantastic winery (Domaine du Castel), and dinner in a Arab Village (Abu Gosh) near the winery. Here are some scenes from the day and night.

My student, M, and I float in the Dead Sea. That’s Jordan in the background.
A panorama of Jerusalem taken from Mt. Scopus. Visible are the old city of Jerusalem, including the Temple Mount.
A panorama from inside the Church of the Holy Sepulcher, showing the structure that encloses the claimed tomb in which Jesus was buried.
The Western Wall of the Second Temple, one of the holiest sites in Judaism. Visible beyond the wall is the Temple Mount, home to the Holiest of Holies in the Judaic tradition (the rock upon which God built the entire Universe) and the Al Aqsa Mosque, third-holiest site in Islam (the site where Muhammad is said to have ascended to Heaven).

Both ends of our table in Abu Gosh, with my fellow workshop participants enjoying a feast after a day of sun and walking and wine.

Workshop Scenes

Many of the participants in the “Hammers and Nails” workshop.
Physicist Daniel Whiteson entertains us with “(Mis-)Communicating Science” during the workshop dinner inside the Koffler Accelerator.
Physicist Kyle Cranmer (NYU) discussing problems in particle physics, statistics, and data science with data scientists during a break in the workshop.

Fun at Weizmann

Besides the workshop, there were other opportunities to do something other than physics from time-to-time. Here are a few.

Selfies with the giants of quantum physics who did their work in the first half of the 20th century. These busts are littered around the physics building at Weizmann and I took a moment to pose with them (and make some faces like theirs…). From top to bottom: Albert Einstein, Wolfgang Pauli, Albert Einstein again, and Neils Bohr.

Kyle Cramner (left), Eilam Gross (right), and I enjoy a fantastic dinner prepared by Eilam and his family at his apartment in North Tel Aviv. I got to meet Eilam’s two daughters and his girlfriend’s son. It was a really awesome evening, and I hadn’t eaten that well in weeks. Kyle and I were wordless with food and exhaustion afterward.

Home Again

I departed Israel on July 31, and it was a long travel day. I was up at 1am to finish packing and pick up the last things in my apartment. My originally scheduled cab did not arrive at 2:30am to pick me up, so I called another one. I arrived at Ben Gurion airport a little after 3am. It took a few hours to get through security, check-in, and more security, and it was a nice 5 hour flight to Frankfurt. I only had a 1 hour layover there, and so I went as fast as I could to catch my connection. But I took a wrong turn and went through the wrong security checkpoint, only to find out the signs pointing to my gate had taken me to the wrong gates! I arrived at the correct gate just 10 minutes before they were to close the door for boarding, but I made it. Our flight landed in Dallas 45 minutes ahead of schedule. Jodi wasn’t even at the airport when I got out of customs, but she managed to make it a few minutes later and I was able to go home and rest.

Now it’s just nice to be home again. Even if I am horribly jet-lagged again.

This concluded the work-related travel for the summer. I have a small break over the next couple of weeks, then the fall semester begins. And, oh, what a busy one this will be.