Steve Schmidt

About

Scientist, mathematician, engineer, lecturer — if there is a novel unanswered problem or research area, I'm excited for it. Academically, I received a second M.S. in Computer Science from the University of Chicago following my Algebra & Analysis Qualification Exams for my first M.S. in Theoretical Mathematics from DePaul University. The intersection of my experiences is in applying Deep Machine and Reinforcement Learning to decision making in marketing, business, finance and operations management.

My work is guided by a "systems-first" philosophy — a problem-agnostic thought process proven across seemingly unrelated domains, from DARPA-scale cyber defense to Nobel Prize-winning FCC auctions to Robotics. As I like to say: "It's the same math that's used to train an unmanned undersea vehicle as we use to personalize your website experience at Nike."

I am always open to looking at data sets of varying size, obscure fields, applying Deep Learning to weird problems and basically any use of algorithms and predictive models to replace our legacy decision processes.

My pursuit is always looking for more folks who are extremely "Driven, Intelligent, Humble, and Chaotic...wherever I can get it" — Jack Donaghy, 30 Rock.

Awards & Recognition

Reinforcement Learning

In the realm of Deep Reinforcement Learning, I have worked on everything from testing learning algorithms in operational settings to working on novel neural network design. RL, loosely, is comprised of an agent existing in an environment — or the math version of controlling Zelda's Link or the infamous Super Mario in their respective environments. The interaction is simply deciding to take defined actions given the current defined state. The training of the policy results from the feedback of experiences from state transitions.

The challenge area I tend to focus on is the problem breakdown. Using state-of-the-art algorithms, can I find novel problems in which to coordinate complex agent control decisions or preferably multi-agent control decisions and situational awareness. The challenge becomes reward shaping, observation space transformations and in many cases hierarchical decision formulations. The second challenge then arises: explainability. In many cases strategy is not defined by the output of activation layers as in academic settings, but rather a defined human-understandable sense. This mapping of continuous state and action pairs along with neural processing is an intriguing path toward what a domain expert may deem as reasonable behavior for an agent.

DEEPMISSION

A customized AI Research Lab built on top of RLlib (OpenAI Gym based). Similar to Single Agent & Multi-Agent Reinforcement Learning frameworks being produced by leading Open Source and Academic Institutions, this platform leverages state-of-the-art & proprietary learning algorithms with the Gym Framework. In addition to connecting algorithms to Gym Environments of Atari Games or custom builds from real time strategy games such as StarCraft II, the platform extends and adds APIs with connections to high TRL operational platforms, systems and planner environments for live, robust, open world agent exploration.

This allows the ability to quickly prototype new learning algorithms within real live operational systems, along with creating sandboxes for researcher scientists and academic partners to jointly develop new low TRL approaches. The platform's highly modular framework allows researchers to quickly create wrapper add-on components for advancing state-of-the-art techniques in Hierarchical Rewards, Empowerment, Intrinsic Motivation, Imitation Learning, and Parallel Learning Implementations.

Experience

Currently, I lead a team for Embodied AI at Boston Dynamics and serve as an Adjunct Professor at Northeastern University teaching graduate Applied Deep Learning. Previously, I served as Director of Applied Science for Enterprise Commerce AI at Nike, Inc., where I oversaw a team developing and optimizing machine learning models for search and recommendation systems — evolving search from lexical matching to natural language models. Prior to Nike, I held research positions as Sr. Principal Scientist at Wayfair, BAE Systems (FAST Labs Division), and Raytheon, working across Internal R&D, DARPA, and AFRL research projects.

Over the course of over a decade of leading dozens of DARPA and industry programs, I have been fortunate enough to work with incredible people — from undergraduates to Nobel Prize recipients — in garnering accolades as a finalist in the 2016 DARPA Grand Challenge, recipient of the 2018 Franz Edelman INFORMS Prize, a 2024 Emmy, and contributing to several journal, conference, and patent publications.

Teaching

Adjunct Professor at Northeastern University's Khoury College of Computer Sciences, where I teach graduate-level Applied Deep Learning and Machine Learning.

Resources

A few things I recommend and reference often:

• Bar Fly Investing, 3rd Edition (2022)
• Goodfellow, Bengio & Courville: Deep Learning
• Sutton & Barto: Reinforcement Learning
• OpenAI Key Papers in Deep RL