Steve Schmidt


Applied Mathematician
Artificial Intelligence

steveschmidt at uchicago dot edu

    Scientist, Mathematician, Engineer, Lecturer...if there is a novel unanswered problem or research area, I'm excited for it! Adacemically, I received a second M.S. in Computer Science from the University of Chicago following my Algebra & Analysis Qualification Exams for my first M.S. in Theoretical Mathematics from DePaul University. The intersection of my experiences is in applying Deep {Machine, Reinforcement} Learning to decision making in marketing, business, finance or operations management. This site is frequently updated to highlight evolving interests, projects and research. Please feel free to contact me via email or LinkedIn to connect on similar interests!

    In the realm of Deep Reinforcement Learning (RL), I have worked on everything from testing learning algorithms in operational settings to working on novel neural network design. RL, loosely, is comprised of an agent, existing in an environment - or the math version of controlling Zelda's Link or the infamous Super Mario - in their respective environments. The interaction is simply deciding to tak defined actions (based on your controller) or decision space, given the current defined state (monitor, graphics, sound, etc). The training of the policy (agent playing the game), results from the feedback (rewards) of the experiences from state transitions.

    The challenge area I tend to focus on is the problem breakdown. Using state-of-the-art algorithms, can I find novel problems (environments) in which to coordinate complex agent control decisions or preferably multi-agent control decisions and situational awarenesss. The challenge becomes reward shaping (incentives), observation space transformations and in many cases hierarchical decision formulations. At that point, tuning of algorithms and watching tensorboard do its thing becomes an overnight process. The second challenge then arises, which is explainability. In many cases strategy is not defined by the output of the activation layers such as in academic settings, but rather a defined human understandable sense. This mapping of continuous state and action pairs along with neural processing is an intriguing path toward what a domain expert may deem as reasonable behavior for an agent. Rinsing and repeating this process is where all the fun is. Mixtures of analytics, deep learning for computer vision, classification or ensembles of algorithmic processes help turn observations and rewards from raw and trivial to complex and innovative.

    Personal rewards in the mathematical realm...

    2024 EMMY Award (National Academy of Television Arts & Sciences) for creativity and engineering of the FCC's Broadband Incentive Auction
    2023 US Patent Award US/2023/0004767 A1 - Radio Frequency Environment Awareness With Explainable Results
    2021 INFORMS Operations Research & Analytics Prize (Wayfair Data Science)
    2020, 2019 BAE Systems Chairmanss Award Nominations (2020 Innovation of the Year Award - Mindful™)
    2018 INFORMS Franz Edelman Award (Operations Research for the novel 2016 FCC Incentive Auction)
    2016 DARPA Grand Challenge Finalist (Deep Red)

    DEEPMISSION: A Deep Reinforcement Learning Platform

    A customized AI Research Lab built on top of RLlib (OpenAI Gym based). Similar to Single Agent & Multi-Agent Reinforcement Learning (MARL) frameworks being produced by leading Open Source and Academic Institutions, this platform leverages the library of state of the art & proprietary learning algorithms with the Gym Framework. However, in addition to connecting algorithms to Gym Environments of Atari Games or custom builds from real time strategy games such as StarCraft II, The Platform extends and adds API's with connections to high TRL operational platforms, systems and planner environments for live, robust, open world agent exploration. This allows the ability to quickly prototype new learning algorithms within real live operational systems, along with creating sandboxes for researcher scientists and academic partners to jointly develop new low TRL approaches. This scalable architecture also can spin up and down from laptops to server clusters to massively parallel DoD cloud infrastructures. The Platform's highly module framework allows researchers to quickly create wrapper add-on components for advancing state of the art techniques in Hierarchical Rewards, Empowerment, Intrinsic Motivation, Imitation Learning, Parallel Learning Implementations or leveraging advancements in tangential work in Computer Vision or Explainable AI while plugging directly into the latest operational systems.

    Deep Cuts (aka some cool things I've worked on in Deep Learning)

    * Deep RL for Series of Pursuit & Evasion Games - Single, Multi, Hierarchical Forms
    * Deep RL for Improving RF Classification
    * Deep RL for Multi-Agent Hierarchical Command & Control with Low Level Tactical Movement
    * Deep RL for Force on Force Simulation Studies
    * Custom Architectures of Deep Dialated Convolutional Networks for Signal and Image Classification
    * Autoencoders for Signal Reconstruction
    * Generative Adversarial Network Plus Variational Autoencoder for Robustness Studies
    * Generative Adversarial Networks for Image Misclassification
    * Introspection Clustering Over Deep Neural Network Activations
    * Convolutional Neural Network in Gated Series for Intrusion & Anamoly Detection


    PUBLICATIONS, PRE PRINTS & CONFERENCES

    Multi-Agent Reinforcement Learning Approaches to RiftNet RF Fingerprint Enhancement, 14th ACM Conference on Security and Privacy in Wireless and Mobile Networks, May 2021
    RiftNeXt: Explainable Deep Neural RF Scene Classification, 14th ACM Conference on Security and Privacy in Wireless and Mobile Networks, May 2021 (patent pending)
    Reinforcement Learning Approach to Speed-Overmatched Pursuit Games with Uncertain Target Information, 5th Annual Naval Applications of Machine Learning (NAML 2021)
    Uncovering Strategies and Commitment through Machine Learning System Introspection, Submitted for Review.
    Machine Learning for NetFlow Anomaly Detection with Human Readable Annotations. IEEE Transactions on Network and Service Management, April 2021
    Automatic Knowledge Extraction with Human Interface, arxiv:2104.04415



    Professional Life
    In my professional career I've held various roles in fields ranging from Marketing to Scientific Research, always hoping to have one foot in the Board Room and one foot in the Lab. Currently, I lead Machine Learning at Nike, leveraging my work as a Staff Scientist at Wayfair, working on frameworks and algorithms for Search & Recommendations. This ranges from Personalization & Sort Optimization to LLMs for Search and longer term novel algorithmic (Deep RL) solutions. Previously, I held a similar role at BAE Systems (FAST Labs Division) as a Sr. Principal Research Scientist in Artificial Intelligence for Internal R&D, or DARPA and AFRL low Technology Readiness Level (TRL) research projects. On these projects I have served as Principal Investigator, researcher, technical lead and idea generator leading to funding, multi-year contracts, publications and patent filings. I am always open to looking at data sets of varying size, obscure fields, applying Deep Learning to weird problems and basically any use of algorithms and predictive models to replace our legacy decision processes. In my free time, I enjoy reading other's research in related fields such as behavioral economics, finance and consult on many topics to many fields in areas of applied mathematics as well as serve as an Adjunct Professor at the College of Charleston (Data Science).







    Project Task Forces
    *FCC Incentive Auction - We won the Edelman Prize for Operations Research & made the Government $30+ Billion...and eventually an EMMY Award!
    *DARPA Cyber Grand Challenge - Took 4th out in Vegas (of 11), first introduction to large scale competitive gaming analytics in simulation!



    Personal Life
    Outside of work for regular pay, my personal tastes mimick my professional tastes as there are always a few projects scattered around my monitors. Most of these fall into Deep Learning, Reinforcement Learning or Machine learning for marketing, advertising, finance or operations management. As well I enjoy lecturing and giving talks or presentations. It has been a continued pleasure working with a wide range of folks, from high school students to CEO's in figuring out how to make us better.

    My pursuit is always looking for more folks who are extremely "Driven, Intelligent, Humble, and Chaotic...wherever I can get it" - Jack Donaghy, 30 Rock.

    Academic Life
    Computer Science: Computational Mathematics (Machine Learning/Numerical Methods)
    Mathematics: Linear Algebra and Mathematical Modeling
    Marketing: Consumer Behavior

    Educational Highlights

      Computer Science & Research

        My previous research was at Argonne National Laboratory/University of Chicago in the field of High Performance Computing. My research title was "Developing Efficient Methods For Parallel Data Redistribution On Hybrid Architectures", with my advisor: Dr. Andrew Siegel, who is a tenured Research Director at Argonne's Center for Exascale Simulation of Advanced Reactors (CESAR). The main goal was working to advance the performance/analysis of the C/MPI based application library The Memory-Aware Data Redistribution Engine (MADRE). This entails looking into the integration of OpenMP into the MADRE Library for shared memory analysis, as well as investigating the Pinar-Hendrickson/Cyclic Scheduler Algorithms analysis for varied input maps, varied memory constraints.
        Aside from adventures in High Performance Computing in education, my focus was on computational data analysis, machine learning and predictive modeling (Clustering, Support Vector Machines, Recommenders, Neural Networks, etc).

      Mathematics

        From 2011-2013, I completed graduate work in Theoretical Mathematics, however, I probably/actually was more interested in the applications than the theory. I took several courses in Numerical Methods and Mathematical Modeling and continue to look at numerical recipes for applications. The uses of various topics in interpolation and applications of single value decomposition are incredible. Linear Algebra of course was another favorite subject of mine as well as where it spawned into Group Theory where again, one can see immediate applications in data structures. Again a complete list of course work, topics & highlighted projects can be found on my Academia page. I'm looking forward to the day where Measure Theory pops up in application...

      • American Mathematical Society

    Consulting & Extracurricular

    Adjunct Professor - Machine Learning @ Northeastern University, Data Science @ College of Charleston

      Guest Lecturer - DePaul University Driehaus School of Business
          Spent classes lecturing and working with MBA students in the area of Creativity & Innovation in Business, this was incredibly fun to mix backgrounds and ideas to problem solve in disruptive markets and products.

        South Loop Elementary - Gifted Students Algorithms Program
          One of the most rewarding experiences ever! I worked with the school to put on a 4 week algorithms program working with their 5th-8th grade gifted students to show them fun with mathematical thinking. We started by looking at the rubik's cube and talked about how to solve it with 6 algorithms based on Group Theory. While we didn't use the words abelian or commutative, we practiced the methods a little bit each week and by week 4 a 7th grade student who saw the cube for the first time in his life only weeks prior solved it under 90 seconds! We also looked at Zeno's Paradox and some simple ciphers.

        Web Analytics Consulting - Google Analytics Consultant For Small Businesses
          Consultant to local small businesses of which it would not make sense for them to hire a large agency or full time employee to setup and run website analytics. I have a Google Analytics Individual Certification and in installing and using Google Analytics acted as their online marketing arm to help them make data driven design and pricing decisions.

        Method - Startup of Startups
          Partnered with area eCommerce/Web Development company where my brother Mike is Director of eCommerce and local Venture Capital firms to research disruptive technology opportunities. After reviewing many of their Labs projects, I acted as CEO of our concept incubator company called Method to work with MBA students at Chicago Universities to research and bring to life ideas using the Lean Start Up methodology.

        Behavior Quantified - Proprietary Data Warehousing Project
          In detecting a problematic scenario in my industry, I decided to create a product called LynchPyn to serve as a sort of dynamic datawarehouse for legacy technologies to map into and report out of. Our project is still in beta and looking for testers. We also ran dashboards, analytics and CRM ETL's into LynchPyn.