Stanford Vision and Learning Lab: Performing Research at the Forefront of Computer Vision, Machine Learning, and Robotics

A Look At The Stanford Vision And Learning Lab

TL; DR: The Stanford Vision and Learning Lab (SVL), spearheaded by Professors Fei-Fei Li, Juan Carlos Niebles, and Silvio Savarese, is a research group working to further theoretical frameworks and practical applications of computer vision. The extensive and diverse lab, powered by students, alumni, research scientists, and visiting scholars, is committed to tackling challenges at the intersection of computer vision, machine learning, and robotics. With its groundbreaking research and the rich pipeline of experienced graduates it produces, SVL is bringing the future of artificial intelligence into view.

To describe Stanford Professor of Computer Science Fei-Fei Li as an expert in her field would be an understatement.

The prolific researcher has contributed to nearly 200 highly respected scientific journals. She’s served as director of Stanford’s Artificial Intelligence Lab. And she invented ImageNet, a visual dataset and benchmarking effort designed to further developments in deep learning and AI.

In recognition of her many efforts to advance the industry, Fei-Fei was named one of the 2016 Great Immigrants by the Carnegie Foundation, following in the footsteps of Albert Einstein, Yoyo Ma, and Sergey Brin.

In 2017, during her sabbatical from Stanford, she was hired as Chief Scientist of AI/ML at Google Cloud. There, she helped grow the company’s cloud computing business through machine learning.

Fei-Fei Li, Juan Carlos Niebles, and Silvio Savarese

A group of renowned leaders in computer science direct the lab.

Considering Fei-Fei’s proficiency and experience, it’s easy to see why the AI visionary, along with renowned Professors Juan Carlos Niebles and Silvio Savarese, were prepared to found the Stanford Vision and Learning Lab (SVL). The groundbreaking research organization is working to further knowledge in computer vision — a complex subset of artificial intelligence that involves teaching computers to register objects within images.

“We are extremely learning-oriented and interested in how cutting-edge, compute-heavy, and data-heavy resources can drive progress on the greatest challenges — as Fei-Fei accomplished with ImageNet early on,” said Andrey Kurenkov, Doctoral Candidate at Stanford and Communications Specialist for SVL.

The lab recruits forward-thinking students like Andrey, as well as alumni, visiting scholars, and professionals worldwide, to help formulate theoretical foundations and discover practical applications at the intersection of computer vision, machine learning, and robotics. By producing a constant stream of industry and academic talent as well as groundbreaking research, SVL is working to shape the future of the artificial intelligence industry at large.

Tackling Fundamental Problems in Computer Vision Research

The research organization began as two separate labs, the Stanford Vision Lab and the Stanford Computational Vision and Geometry Lab, before merging to become the Stanford Vision and Learning Lab.

“Fei-Fei Li, Juan Carlos Niebles, and Silvio Savarese combined their efforts to create one large lab that’s focused on vision and its related applications,” Andrey said. “We do a lot of computer vision research, but now we have subgroups that focus on things like 3D modalities (which are useful in self-driving cars), robotics, and healthcare.”

Andrey told us that SVL’s mission is largely the same as that of the larger artificial intelligence research community, albeit focused on computer vision. “Most labs of our type are trying to develop and push forward technologies to ultimately have a beneficial impact on the world,” he said.

An image of Jackrabbot

The Stanford Jackrabbot is a self-navigating delivery cart capable of carrying small payloads.

For example, SVL developed the Jackrabbot — a self-navigating electric delivery cart designed to operate in spaces typically designated for pedestrians — to replace or augment human efforts. The autonomous agent was programmed to understand and anticipate human motion in public areas, such as the Stanford campus, where the technology is being tested.

On the healthcare front, SVL is collaborating with the Stanford University School of Medicine to explore how computer vision and machine learning can impact clinical outcomes, health improvements, and the academic knowledge base. Through the Partnership in AI-Assisted Care (PAC), the two groups are working on several initiatives designed to support intelligent hand hygiene, intelligent senior well-being, burn excision, and activity detection in intensive care units.

Ultimately, the goal is to leverage artificial intelligence to improve healthcare while working to eliminate cost barriers.

Fueling Future Research Through Robust Datasets

In addition to highly cited research and cutting-edge development, Andrey said SVL is known for the comprehensive datasets it provides to the academic community.

The lab’s most well-known database, ImageNet, was actually established well before SVL’s formation, but it laid the foundation for advances in visual recognition. The project is organized based on the hierarchy established in WordNet, a database of the English language that has use cases in computational linguistics and natural language processing. ImageNet currently features only nouns, but thousands of images are used to depict each node of the hierarchy.

Associated with the project is the ImageNet Large Scale Visual Recognition Challenge (ILSVRC), a competition in which participants identify objects within an image so they can be classified and annotated. The event, now hosted solely by Kaggle, is designed to ultimately help computers identify subtle differences in photos while furthering scientific knowledge in object detection and classification.

The Visual Genome

The Visual Genome is an initiative connecting structured image concepts to language.

Datasets created after the formation of SVL include The Visual Genome, which helps connect structured image concepts with human language. The ongoing effort is currently made up of millions of image descriptions, objects, attributes, relationships, and question-answer pairs. This type of information is needed for computers to successfully perform cognitive tasks that involve reasoning with the visual world.

Another large-scale database, ObjectNet3D, is being used to help computers recognize 3D objects within 2D images. This information could pave the way for future research in robotics, autonomous driving, and augmented reality.

In addition, PAC is working on a dataset of segmented and annotated burn images known as BURNED. It consists of more than 1,000 images of burns outlined by plastic surgeons and labeled based on depth. Researchers plan to use the data to build an automated visual system capable of predicting burn severity and spatial outline in patients, improving treatment decisions.

Contributions from the Study Body and Alumni

Out of all the challenges SVL tackles, Andrey said that attracting talent isn’t one of them. Fei-Fei Li, Juan Carlos Niebles, and Silvio Savarese are all renowned and respected professors who have made significant impacts in their respective areas of study, and they’re continually working on a diverse and engaging set of projects.

“In many cases, students join the lab while pursuing their master’s degree, or even as an undergrad to be part of a larger project,” Andrey said. “New students can gradually take on more work and become a substantial part of the research, and that’s really appealing.”

Lab members on a retreat

In addition to groundbreaking research, SVL produces a constant stream of industry and academic talent.

Students who work in the lab frequently enter academia on a professional level or secure roles in prominent tech companies. Former lab participant Andrej Karpathy, who focused on deep learning and artificial neural networks while obtaining his doctorate at Stanford, is now Senior Director of Artificial Intelligence at Tesla.

Other distinctive alumni include Timnit Gebru, a research scientist on Google’s Ethical AI team; Jia Li, who served as Head of R&D at Google Cloud AI; and Olga Russakovsky, now Assistant Professor at the Department of Computer Science at Princeton University.

“The number of people who graduate from the lab and end up becoming professors or important industry figures is pretty substantial,” Andrey said. “It goes to show that we are feeding the larger academic community.”

After several years of operating in its current, combined form, Andrey said SVL is hitting its stride.

“Looking forward, our leaders are interested to see whether we’ll start seeing even more benefits in terms of collaboration by the nature of our lab being so large and encompassing so many different types of work,” he said.