Tuesday, December 11, 2012

The Big NIPS Post

2012 was my first year at NIPS, and I'm still getting my legs when it comes to ML, so my impressions are very general. For other takes on NIPS, see Hal Daume's post (a veteran NIPS-goer), and my advisor Suresh Venkatasubramanian's post (a newbie to NIPS like myself).


Monday is tutorial day at NIPS, and I really only got something that I wanted out of one tutorial, by Joel Tropp. Joel did a great presentation on random matrices and distilled the very confusing body of work out there to a nice, clean idea. Basically, break random matrices down into a sum of simpler, independent random matrices, then apply concentration bounds on the sum. He worked through several examples and the ideas were easy to understand.


Scott Aaronson's talk was great, not surprisingly. Scott talked about "Quantum Information and the Brain," such as what QC's will be good at, and whether the human brain is or isn't probably a quantum computer. Scott thinks that the most likely place that QC's will be employed, when we get them working at a large enough scale to be useful, is in simulating QM, and probably not as much in other applications. He underscored the fact that most people have a fundamental misunderstanding of how QC's work, and that most QC theorists think that NP is not a subset of BQP (the complexity class that covers quantum algorithms). 

Scott doesn't believe that the human brain is a QC, and gave one of the most compelling arguments in my mind that it isn't. That is, we know what QC's are good at: factoring integers, discrete logarithms, and simulating QM. There isn't any evidence to date that humans are any good at any of these things. He says that the idea that the brain might be a QC is usually the product of thinking, "brain is mysterious, QM is mysterious, therefore brain = QC." 

I tend to agree with him, and it seems that the human brain is an exceptionally good pattern recognizer and identity factory (read "self-deluder"), but not really a great general-purpose computer, especially a quantum one.

Wednesday & Thursday

I'm going to freely admit that I didn't get much out of these days. As I get more accustomed to the material, I'll probably have a lot more to explore. There were a lot of posters to look at, and it was cool to see some kernel methods, metric learning, even Riemannian techniques.


Friday and Saturday are the workshop days, and I felt like I got a lot out of these. I visited the two social network workshops, and there was a lot of fun stuff to look at. The invited speakers were all great, and it was interesting to see more algorithms-related stuff going on along with ML.


Saturday was all about the optimization workshop for me. Pablo Parillo and Francis Bach both had great talks, and this is where I presented our work (arXiv) and poster.  We got a lot of great feedback from the audience and a lot of directions to follow.

General Impressions

Primary impression: NIPS is exhausting! There's tons of stuff to look at and do. This year was definitely all about deep learning, so I'm going to make sure I know the basics before next year. 

Venue: They can definitely do better than S. Lake Tahoe CA/Stateline NV. The conference was at Harvey's and Harrah's on the NV side, and you're basically stuck at these casinos and the surrounding blocks if you don't want to fork out a bunch for a cab. 

There's not a whole lot of options around for food, but definitely go try Super Taco about a block from Harvey's. Great tortas and a really, really nice change from the casino options -- the owner is a great guy too (only outdoor seating, but a quick walk back to Harvey's). Don't go to the Cabo Wabo in Harvey's. It's a very pathetic tribute to Sammy Hagar and has overpriced, mediocre-to-crappy food. If you're into the crappy-coffee-steak-and-eggs thing, there's a diner in the basement of Harrah's underneath the conference floor.

I didn't ski, so you'll have to ask someone else about that. I was surprised that anyone was skiing on the rocks and ice I saw on the mountains, but I'm from Utah, so I guess I have a higher standard. 

Biggest gripe about the venue: you're stuck. It's really not worth it to rent a car, because you'd have to chain your tires to get over the mountains, and I have no idea what options Reno is going to have for rentals. It's not a big city. So you have to take a shuttle to get between Reno and Tahoe, and that takes as long or longer than the leg between Salt Lake and Reno. You also have to make that extra reservation, and account for about 2 extra hours to get to the airport.

Facilities: Nice rooms, but jeez, gimme more outlets! My con roomie and I were stuck in weird spots in the room to get power. Bring an extension cord or power strip. They had an old buffet room converted into the "Internet Cafe." The wifi reception was the best in this area and they had tables with lots of power strips, which was nice. This was the best alternative to getting reception in the hotel room. We stayed at Harvey's, and the beds, chairs, bathroom, and shower were all good. Comfy place to crash. HD televisions but no HD cable.

Like Hal said in his post, the walls were like paper. This made it really hard to listen to a quiet invited speaker while there was conversation in the hallways or a loud speaker in the next room. Also not a lot of ventilation for crowded rooms.


Overall, I definitely wouldn't mind coming back to NIPS, but I look forward to when they move the venue.  I also don't know if I have the energy for the whole conference. It's big!