About Me

I am Dr. Nicholas Waytowich, a Machine Learning Research Scientist with the US Army Research Laboratory, where my primary role revolves around human-guided Machine Learning. Currently, I take pride in being the lead scientist for the ARL’s Human-Guided Machine Learning Branch. Along with my dedicated team of researchers, we are at the forefront of devising innovative algorithms for human-guided AI/ML. My research shines a spotlight on harnessing human feedback through human-in-the-loop machine learning and reinforcement learning. My diverse interests span across human-guided machine learning, deep reinforcement learning, large-language models, human-agent teaming, and human-robot collaboration.

Before my stint with the Army Research Laboratory, I had the privilege of being a postdoctoral fellow under the esteemed Paul Sajda at Columbia University in the Laboratory for Intelligent Imaging and Neural Computing. During this period, I delved deep into creating novel brain-computer interfaces. This pursuit was a continuation of my Ph.D. journey in Biomedical Engineering from Old Dominion University, where I worked closely with Dean Krusienski on practical brain-computer interface applications.

In addition to my research roles, I’ve been sharing my knowledge as an Adjunct Professor at both the University of Maryland Baltimore County (UMBC) and Anne Arundel Community College (AACC), where I teach Introduction to Machine Learning as well as other engineering and computer science courses.

Having had the honor to present my findings at prominent AI/ML conferences like AAAI, ICML, NeurIPS and AAMAS, I remain committed to pushing the boundaries of what’s possible in my field.

My CV is located here.
My Google Scholar page is here

Publications

PhD Dissertation:

Development of a Practical Visual Evoked Potential Based Brain-Computer Interface
Nicholas Waytowich
ODU Digital Comms, 2015
PDF

Peer-Reviewed Publications:

Investigating the mission impact of non-kinetic variables in the operational environment MR Mittrick, J Richardson, VG Goecks, JZ Hare, N Waytowich Artificial Intelligence and Machine Learning for Multi-Domain Operations, 2024

Adversarial Attacks on Reinforcement Learning Agents for Command and Control A Dabholkar, JZ Hare, M Mittrick, J Richardson, N Waytowich arXiv preprint arXiv:2405.01693, 2024

Scalable interactive machine learning for future command and control A Madison, E Novoseller, VG Goecks, BT Files, N Waytowich, A Yu 2024 International Conference on Military Communication and Information, 2024

COA-GPT: Generative pre-trained transformers for accelerated course of action development in military operations VG Goecks, N Waytowich 2024 International Conference on Military Communication and Information, 2024

Rating-based reinforcement learning D White, M Wu, E Novoseller, VJ Lawhern, N Waytowich, Y Cao Proceedings of the AAAI Conference on Artificial Intelligence 38 (9), 10207, 2024

On games and simulators as a platform for development of artificial intelligence for command and control VG Goecks, N Waytowich, DE Asher, S Jun Park, M Mittrick, J Richardson The Journal of Defense Modeling and Simulation 20 (4), 495-508, 2023

DIP-RL: Demonstration-Inferred Preference Learning in Minecraft E Novoseller, VG Goecks, D Watkins, J Miller, N Waytowich arXiv preprint arXiv:2307.12158, 2023

Disasterresponse-GPT: Large language models for accelerated plan of action development in disaster response scenarios VG Goecks, NR Waytowich arXiv preprint arXiv:2306.17271, 2023

Towards solving fuzzy tasks with human feedback: A retrospective of the minerl basalt 2022 competition S Milani, A Kanervisto, K Ramanauskas, S Schulhoff, B Houghton, N Waytowich arXiv preprint arXiv:2303.13512, 2023

Starcraftimage: A dataset for prototyping spatial reasoning methods for multi-agent environments S Kulinski, NR Waytowich, JZ Hare, DI Inouye Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

Learning Flight Control Systems from Human Demonstrations and Real-Time Uncertainty-Informed Interventions P Ganesh, JH Ramos, VG Goecks, J Paquet, M Longmire, NR Waytowich IFAC-PapersOnLine 56 (2), 6265-6272, 2023

Negative Obstacle Traversal of Physical Ground Robots via Imitation Learning Based Control
Brian Cesar-Tondreau, Garrett Warnell, Kevin Kochersberger and Nicholas Waytowich
Robotics and Autonomous Systems, 2023

Permtl: A multi-task learning framework for skilled human performance assessment I Ghosh, A Chakma, SR Ramamurthy, N Roy, N Waytowich 2022 21st IEEE International Conference on Machine Learning and Applications, 2022

Learning to guide multiple heterogeneous actors from a single human demonstration via automatic curriculum learning in StarCraft II
Nicholas Waytowich, James Hare, Vinicius Goecks, Mark Mittrick, John Richardson, Anjon Basak, and Derrik Asher
SPIE: Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications IV, 2022
PDF

A Flexible Software-Hardware Framework for Brain EEG Multiple Artifact Identification
Mohit Khatwani, Hasib-Al Rashid, Hirenkumar Paneliyua, Mark Horton, Houman Homayhoun, Nicholas Waytowich, David Hairson, and Tinoosh Mohsenin
Handbook of Biochips, 2022
PDF

Mobile manipulation leveraging multiple views
David Watkins-Valls, Peter Allen, Henrique Maia, Madhavan Seshadri, Jonathan Sanabria and Nicholas Waytowich
2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
PDF

On games and simulators as a platform for development of artificial intelligence for command and control
Vinicius G Goecks, Nicholas Waytowich, Derrik Asher Song Park, Mark Mittrick, John Richardson, Manuel Vindiola, Anne Logie, Mark Dennison, Theron Trout and others
The Journal of Defense Modeling and Simulation, 2022
PDF

E2hrl: An energy-efficient hardware accelerator for hierarchical deep reinforcement learning
Aidin Shiri, Uttej Kallakuri, Hasib-Al Rashid, Bharat Prakash, Nicholas Waytowich, Tim Oates and Tinoosh Mohsenin
ACM Transactions on Design Automation of Electronic Systems (TODAES), 2022
PDF

Retrospective on the 2021 BASALT Competition on Learning from Human Feedback
Rohin Shah, Steven Wang, Cody Wild, Stephanie Milani, Anssi Kanervisto, Vinicius Goecks, Nicholas Waytowich, David Watkins-Valls, Bharat Prakash, Edmund Mills and others
Proceedings of Machine Learning Research (PLMR), NeurIPS 2021 Competitions and Demonstrations Track, 2022
PDF

Utility of doctrine with multi-agent RL for military engagements
Anjon Basak, Erin Zaroukian, Kevin Corder, Rolando Fernandz, Christopher Hsu, Piyush Sharma, Nicholas Waytowich and Derrik Asher
SPIE: Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications IV, 2022
PDF

Towards fully autonomous negative obstacle traversal via imitation learning based control
Brian Cesar-Tondreau, Garrett Warnell, Kevin Kochersberger, and Nicholas Waytowich
MDPI: Robotics, 2022
PDF

TagTeam: Towards wearable-assisted, implicit guidance for human-drone teams
Kasthuri Jayarajah, Aryya Gangopadhyay, Nicholas Waytowich
Proceedings of the 1st ACM Workshop on Smart Wearable Systems and Applications, 2022
PDF

An optimization framework for efficient vision-based autonomous drone navigation
Mozhgan Navardi, Aidin Shiri, Edward Humes, Nicholas Waytowich and Tinoosh Mohsenin
IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS), 2022
PDF

Efficient Language-Guided Reinforcement Learning for Resource-Constrained Autonomous Systems
Aidin Shiri, Mozhgan Navardi, Tejaswini Manjunath, Nicholas Waytowich and Tinoosh Mohsenin
IEEE Micro, 2022
PDF

Towards an interpretable hierarchical agent framework using semantic goals
Bharat Prakash, Nicholas Waytowich, Tim Oates, and Tinoosh Mohsenin
NeurIPS Workshop on Language and Reinforcement Learning (LaReL), 2022
PDF

Combining learning from human feedback and knowledge engineering to solve hierarchical tasks in minecraft
Vinicius Goecks, Nicholas Waytowich, David Watkins and Bharat Prakash
AAAI MAKE, 2021
PDF

Human-autonomy teaming for the tactical edge: the importance of humans in artificial intelligence research and development
Kristin Schaefer, Brandon Perelman, Joe Rexwinkle, Jonroy Canady, Catherine Neubauer, Nicholas Waytowich, and others
Systems Engineering and Artificial Intelligence, 2021
PDF

An energy-efficient hardware accelerator for hierarchical deep reinforcement learning
Aidin Shiri, Bharat Prakash, Arnab Mazumder, Nicholas Waytowich, Tim Oates and Tinoosh Mohsenin
IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), 2021
PDF

A Hardware Accelerator for Language-Guided Reinforcement Learning Aidin Shiri, Arnab Mazumder, Bharat Prakash, Houman Homayoun, Nicholas Waytowich and Tinoosh Mohsenin
IEEE Design & Test, 2021
PDF

A flexible multichannel eeg artifact identification processor using depthwise-separable convolutional neural networks Mohit Khatwani, Hasib-Al Rashid, Hirenkumar Paneliya, Mark Horton, Nicholas Waytowich, David Hairston and Tinoosh Mohsenin
ACM Journal on Emerging Technologies in Computing Systems (JETC), 2021
PDF

An Energy Efficient EdgeAI Autoencoder Accelerator for Reinforcement Learning Nitheesh Manjuath, Aidin Shiri, Morteza Hosseini, Bharat Prakash, Nicholas Waytowich and Tinoosh Mohsenin
IEEE Open Journal of Circuits and Systems, 2021

Combining learning from human feedback and knowledge engineering to solve hierarchical tasks in Minecraft Vinicius G. Goecks, Nicholas Waytowich, David Watkins, Bharat Prakash AAAI MAKE, 2021

Human-autonomy teaming for the tactical edge: the importance of humans in artificial intelligence research and development Kristin E. Schaefer, Brandon Perelman, Joe Rexwinkle, Jonroy Canady, Catherine Neubauer, Nicholas Waytowich, Gabriella Larkin, Katherine Cox, Michael Geuss, Gregory Gremillion Systems Engineering and Artificial Intelligence, 2021

An energy-efficient hardware accelerator for hierarchical deep reinforcement learning Aidin Shiri, Bharat Prakash, Arnab Neelim Mazumder, Nicholas R. Waytowich, Tim Oates, Tinoosh Mohsenin 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), 2021

A Hardware Accelerator for Language-Guided Reinforcement Learning Aidin Shiri, Arnab Neelim Mazumder, Bharat Prakash, Houman Homayoun, Nicholas R. Waytowich, Tinoosh Mohsenin IEEE Design & Test, 2021

A flexible multichannel EEG artifact identification processor using depthwise-separable convolutional neural networks Mohit Khatwani, Hasib-Al Rashid, Hirenkumar Paneliya, Mark Horton, Nicholas Waytowich, W. David Hairston, Tinoosh Mohsenin ACM Journal on Emerging Technologies in Computing Systems (JETC), 2021

An Energy Efficient EdgeAI Autoencoder Accelerator for Reinforcement Learning Nitheesh Kumar Manjunath, Aidin Shiri, Morteza Hosseini, Bharat Prakash, Nicholas R. Waytowich, Tinoosh Mohsenin IEEE Open Journal of Circuits and Systems, 2021

Energy-Efficient Hardware for Language Guided Reinforcement Learning Aidin Shiri, Arnab Neelim Mazumder, Bharat Prakash, Nitheesh Kumar Manjunath, Houman Homayoun, Avesta Sasan, Nicholas R. Waytowich, Tinoosh Mohsenin Proceedings of the 2020 on Great Lakes Symposium on VLS, 2020

The Autoregressive Linear Mixture Model: A Time-Series Model for an Instantaneous Mixture of Network Processes Addison W. Bohannon, Vernon J. Lawhern, Nicholas R. Waytowich, Radu V. Balan IEEE Transactions on Signal Processing, 2020

Learning your way without a map or compass: Panoramic target driven visual navigation D. Watkins-Valls, J. Xu, Nicholas Waytowich, P. Allen IEEE/RSJ International Conference on Intelligent Robots and Systems, 2020

Yesterday’s Reward is Today’s Punishment: Contrast Effects in Human Feedback to Reinforcement Learning Agents D. Ramesh, AZ Liu, AJ Echeverria, JY Song, Nicholas Waytowich, WS Lasecki Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, 2020

A Narration-based Reward Shaping Approach using Grounded Natural Language Commands Nicholas Waytowich, Sean L. Barton, Vernon Lawhern, Ethan Stump, Garrett Warnell International Conference on Machine Learning (ICML) Workshop on Imitation, Intent and Interaction, 2019

Grounding Natural Language Commands to StarCraft II Game States for Narration Guided Reinforcement Learning Nicholas Waytowich, Sean L. Barton, Vernon Lawhern, Ethan Stump, Garrett Warnell Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, SPIE 2019, 2019

On the Use of Deep Autoencoders for Efficient Embedded Reinforcement Learning Bharat Prakash, Mark Horton, Nicholas Waytowich, William David Hairston, Tim Oates, Tinoosh Mohensin March 2019

Improving Safety in Reinforcement Learning Using Model-Based Architectures and Human Interventions Bharat Prakash, Mohit Khatwani, Nicholas Waytowich, Tinoosh Mohensin March 2019

Measuring Collaborative Emergent Behavior in Multi-agent Reinforcement Learning Sean L. Barton, Nicholas Waytowich, Erin Zaroukian, Derrik Asher 2019

Efficiently Combining Human Demonstrations and Interventions for Safe Training of Autonomous Systems in Real-Time Vinicius G. Goecks, Gregory M. Gremillion, Vernon J. Lawhern, John Valasek, Nicholas R. Waytowich Association for the Advancement of Artificial Intelligence, 2019

Compact Convolutional Neural Networks for Classification of Asynchronous Steady-state Visual Evoked Potential Nicholas Waytowich, Vernon Lawhern, Javier Garcia, Jennifer Cummings, Josef Faller, Paul Sajda, Jean Vettel Journal of Neural Engineering, 2018

Cycle-of-Learning for Autonomous Systems from Human Interaction Nicholas R. Waytowich, Vinicius G. Goecks, Vernon J. Lawhern AAAI Fall symposium on AI-HRI, 2018

Deep TAMER: Interactive Agent Shaping in High-Dimensional State Space Garrett Warnell, Nicholas Waytowich, Vernon Lawhern, Peter Stone Association for the Advancement of Artificial Intelligence, 2018

Optimization of Checkerboard Spatial Frequencies for Steady-State Visual Evoked Potential Brain-Computer Interfaces Nicholas Waytowich, Yusuke Yamani, Dean Krusienski IEEE Transactions on Neural Systems and Rehabilitation Engineering, June 2017

Development of an Extensible SSVEP-BCI Software Platform and Application to Wheelchair Control Nicholas Waytowich, Dean Krusienski Neural Engineering (NER), 2017 6th International IEEE/EMBS Conference, 2017

EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces Vernon Lawhern, Amelia Solon, Nicholas Waytowich, Steven Gordon, Chou Hung, Brent Lance Journal of Neural Engineering, November 2016

Spectral Transfer Learning using Information Geometry for a User-Independent Brain-Computer Interface Nicholas Waytowich, Vernon Lawhern, Addison Bohannon, Kenneth Ball Frontiers of Neuroscience: Neuroprosthetics, September 2016

Cortically Coupled Computing: A New Paradigm for Synergistic Human-Machine Interaction Sameer Saproo, Josef Faller, Victor Shih, Nicholas Waytowich, Addison Bohannon, Vernon Lawhern, Brent Lance, David Jangraw IEEE Computer, September 2016

Multiclass Steady-State Visual Evoked Potential Frequency Evaluation Using Chirp-Modulated Stimuli Nicholas Waytowich, Dean Krusienski IEEE Transactions on Human-Machine Systems, February 2016

Discriminative Feature Extraction via Multivariate Linear Regression for SSVEP-based BCI Haiqiang Wang, Yu Zhang, Nicholas Waytowich, Dean Krusienski, Guoxu Zhou, Jing Jin, Xingyu Wang, Andrzej Cichocki IEEE Transactions on Neural Systems and Rehabilitation Engineering, February 2016

Unsupervised Adaptive Transfer Learning for Steady-State Visual Evoked Potential Brain-Computer Interfaces Nicholas Waytowich, Josef Faller, Javier Garcia, Jean Vettel, Paul Sajda IEEE International Conference on Systems, Man and Cybernetics (SMC), October 2016

Collaborative Image Triage with Humans and Computer Vision Addison Bohannon, Nicholas Waytowich, Vernon Lawhern, Brian Sadler, Brent Lance IEEE International Conference on Systems, Man and Cybernetics (SMC), October 2016

Spatial Decoupling of Targets and Flashing Stimuli for Visual Brain-Computer Interfaces Nicholas R. Waytowich, Dean J. Krusienski Journal of Neural Engineering, April 2015

Novel Characterization of the Steady-State Visual Evoked Potential Spectrum of EEG Nicholas R. Waytowich, Dean J. Krusienski BrainKDD: International Workshop on Data Mining for Brain Science, 2014

The Challenges of Using Scalp-EEG Input Signals for Continuous Device Control G.J. Johnson, Nicholas R. Waytowich, Dean J. Krusienski Foundations of Augmented Cognition. Directing the Future of Adaptive Systems, 2011

Robotic Application of a Brain-Computer Interface to Staubli TX40 Robots - Early Stages Nicholas R. Waytowich, A. Henderson, Dean J. Krusienski, D.J. Cox World Automation Congress (WAC), 2010