Neural

David Wallace Croft

2012 May 30 Wed



My current research focuses on Computational Neuroscience (CNS), particularly Spiking Neuronal Networks (SNNs). I am also very interested in Computational Neuroethology (CNE) which combines my experience in virtual reality graphics programming and my growing interest in robotics. Previously I studied spike-driven Neuroprosthetics and Artificial Neural Networks (ANNs).


Chronology

This chronology provides a history of my neural research in academia and industry. I also include some of my work on Java graphics programming as it relates to my interest in virtual learning environments and CNE. For future additions to the chronology, please see the CroftSoft Update weblog.

  • 2011 - Present
    Event Related Neuronal Simulation Tool
    Updating Java-based CNS simulator for Johns Hopkins University.

  • 2011-11-14
    CroftSoft IntFire v1.1
    Updated to include exponential decay current synapses.

  • 2011-11-04
    Exponential Decay Current Synapses
    Tutorial slideshow. Presented to the Atzori Lab.

  • 2011-10-07
    Integrate-and-Fire
    Tutorial slideshow on integrate-and-fire neuron models. Presented to the Atzori Lab.

  • 2011-10-07
    IntFire
    An animated interactive Java Swing applet simulating the Integrate-and-Fire spiking neuron model.

  • 2011-08-19
    Hodgkin-Huxley
    Tutorial slideshow on the Hodgkin-Huxley equations. Presented to the Atzori Lab and the Dallas CNS Study Group.

  • 2011-08-19
    Taylor Series
    Tutorial slideshow on the Taylor Series equation as it relates to numerical integration. Presented to the Atzori Lab and the Dallas CNS Study Group.

  • 2011-07-08
    Poisson Distribution
    Tutorial slideshow on the derivation of the Poisson Distribution. Presented to the Atzori Lab and the Dallas CNS Study Group.

  • 2011-05-31
    NE = Quit Signal?
    Tutorial slideshow on norepinephrine as a neuromodulator. Presented to the Atzori Lab.

  • 2010-12-10
    My doctoral research advisor Dr. Cauller retired and I joined the Atzori Lab. I shifted my research focus from CNE to CNS.

  • 2009-07-04
    Concurrent Java Simulations Using Three Phase Update
    Tutorial on writing multi-threaded Java simulation code.

  • 2008-08-29
    Neuro
    Animated interactive Java Swing applet simulating the Hodgkin-Huxley neuron model.

  • 2007 - 2009
    Spike Interface Embodied Virtual Environment
    The purpose of the Spike Interface Embodied Virtual Environment (SIEVE) was to create a virtual environment for simulated neuronal networks in which both effectors and sensors communicated via spikes.

  • 2007 Jan - 2007 Aug
    Unmanned Aerial Vehicle (UAV) Advanced Ground Control Station
    SET Corporation, Arlington, VA
    3D OpenGL graphics, GUI, video, and peer network programming
    UDP discovery and integrated non-blocking I/O HTTP server

  • 2006 Oct - 2006 Dec
    CyBeRev Chatbot Development
    Terasem Movement Inc., Melbourne Beach, FL
    Enabled website with customizable A.I. chatbots
    Created Java Swing HTTP chat client including text-to-speech

  • 2006
    Neuroprosthetic Training Software
    Wrote and submitted a military STTR grant proposal to develop neuroprosthetic training software with a 3D simulation of a prosthetic limb controlled by input from a peripheral nerve spike sensor. The sensor hardware was invented by my research advisor Dr. Cauller.

  • 2006 - 2007
    Infant
    Java Swing desktop application for newborn infant cognition experiments. The software presents visual stimuli and records responses triggered by a pacifier connected to an air pressure transducer via the game port.

  • 2005-12-22
    Research Project: Real-time Simulation and Processing of Peripheral Nerve Spike Activity
    Doctoral first year project report based on my research as a member of the Cauller Lab.

  • 2005 - 2006
    Newt Cyborg
    Animated interactive Java Swing applet simulating a spike-driven neuroprosthetics interface.

  • 2006-02-10
    Real-time Simulation and Processing of Peripheral Nerve Spike Activity
    Slideshow on spike-driken neuroprosthetics simulator. Presented to the Dallas Area Neuroscience Group.

  • 2005 Fall
    Statistics for Psychology
    University of Texas at Dallas, Richardson, TX
    Taught undergraduate course to refresh my knowledge of statistics.

  • 2005 May - Present
    Part-time doctoral student Cognition and Neuroscience
    University of Texas at Dallas, Richardson, TX

  • 2004-10-26
    Correlating Direction of Movement with Spiking Activity
    Research Project Proposal

  • 2004-10-07
    Real-time Control of an Audio Speaker Using Simultaneously Recorded Neurons in the Motor Cortex
    Research Project Proposal

  • 2004 May - 2006 Oct
    Peer-to-Peer Virtual Reality Learning Environments
    Whoola Inc., Richardson, TX

    • Lead Developer and primary author of research status reports
    • Software architecture and network and graphics programming
    • Wrote one of the first 3D graphics COLLADA loaders for Java
    • Wrote a 3D Web browser with fly-through hyperlinks
  • 2004-04-01
    Oral Presentation
    Slideshow presented to Integrative Neuroscience class on research paper Chen-yu Li, Jiang-teng Lu, Chien-ping Wu, Shumin Duan, and Mu-ming Poo (2004) Bidirectional Modification of Presynaptic Neuronal Excitability Accompanying Spike Timing-Dependent Synaptic Plasticity.

  • 2004 - 2010
    Co-founder and officer of the Dallas area chapter of the Society for Neuroscience (SfN), the Dallas Area Neuroscience Group.

  • 2003 May - 2005 May
    M.Sc. Applied Cognition and Neuroscience
    University of Texas at Dallas, Richardson, TX
    Focus: Neuroprosthetics simulation in Java

  • 2003 Spring and Fall
    Computer Game Development
    University of Texas at Dallas, Richardson, TX
    Taught undergraduate course on Java game programming.

  • 2002 - 2004
    Advanced Java Game Programming
    Authored and published book on high-speed Java Swing animation.

  • 1996 - 2003
    Java Developer
    During this period I discontinued most of my neural research as I was swept up into the dot com boom along with everyone else. While in Silicon Valley, I accidentally bumped into two of my former Caltech teaching assistants on separate occasions; both had finished their doctorates in the Computational and Neural Systems program at Caltech and were now developing e-commerce websites. I learned the Java programming language and applied it to game, graphics, software agents, and e-commerce projects.

  • 1996-09-07
    Dice
    Click the mouse and watch a simple neural net learn whether to flee or fight.

  • 1996-08-23
    Insight
    Goblins hunt kobolds in the dark using a neural network.

  • 1996
    BackpropXOR
    Animated interactive Java applet simulating the ANN backpropagation learning rule. This might have been the first ANN Java applet on the Web. I think wrote this while attending the 1996 Neuromorphic Engineering Workshop.

  • 1996-06-24 - 1996-07-14
    1996 NSF Workshop on Neuromorphic Engineering
    Sought feedback on my work on Tanner Research work and discussed the merits of using Neuromorphic VLSI to study neuronal networks. Based on discussions at this conference and my work at Tanner Research, I concluded that high-performance specialized hardware was not the future research bottleneck; instead more emphasis needed to be placed on developing learning algorithms and simulating them on general-purpose computers.

  • 1995 Jun - 1996 Jul
    Systems Engineer
    Tanner Research Inc., Pasadena, CA
    http://www.tanner.com/

    • Design and implementation of parameterizable VLSI layout language software code in C for the automated generation of digital neural network and subthreshold analog VLSI neuromorphic circuits as part of the Neural Network Silicon Compiler research contract. Demonstrated at the 1996 NSF Workshop on Neuromorphic Engineering.
    • Design and fabrication of scalable, programmable, stochastic pulse CMOS VLSI Digital Neural Network Architecture (DNNA) circuitry.
    • Laboratory testing of analog and digital CMOS VLSI chips for speech processing and neural network applications.
    • Documentation of reusable VLSI circuit layout language code components and cell libraries in HTML.
    • Wrote the "Fuzzy Logic Silicon Compiler" government research proposal, identifying low-power analog circuits to be used for Fuzzy Logic processing.

  • 1994 - 1995
    A Depolarizing and Hyperpolarizing Silicon Neuron
    Design, fabrication, and testing of a novel analog VLSI depolarizing-hyperpolarizing neuron with an analog synapse adapted using an integrated learning algorithm with floating gate tunneling and injection. I designed this chip to study STDP; this might have been the first neuromorphic VLSI chip to include the hyperpolarization phase of a spiking neuron. Presented in a talk at the kickoff for the NSF Center for Neuromorphic Systems Engineering at the California Institute of Technology.

  • 1995
    Studied mRNA for spiking neuron channel
    With lab partners, injected mRNA for neural channels into frog oocytes and later observed neural spiking when current was injected.

  • 1994-06-15
    FISL: Hebbian Phase Learning
    STDP research as a graduate student at Caltech.

  • 1994-06-03
    Constructing a Neural Network to Model Self-Timed Circuits
    STDP research as a graduate student at Caltech.

  • 1994-05-21
    HyperNet
    A computer simulation experiment to train a spiking neuronal network to learn a song using STDP.

  • 1994-03-19
    Representational Systems
    Paper that I wrote for a graduate course at Caltech on Theory of Mind.

  • 1994-02-09
    Could a Computer Feel Pain?
    Paper that I wrote for a graduate course at Caltech on Theory of Mind.

  • 1994-01-12
    A Neural Network Primer
    I wrote this to explain ANNs to my newlywed bride Shannon.

  • 1994
    Integrate-and-Fire simulation in MatLab
    Developed simulation software in MatLab for the implementation of a spatiotemporal filter for accurate velocity estimation over a range of spatial frequencies using passive and integrate-and-fire neuron models.

  • 1993-12-20
    Frustrated Inhibited Synapse Learning (FISL) and Forward Inculcated Relief of Excitation (FIRE) Training: Neural Network Learning and Training Algorithms (Ami Pro, HTML, RTF, text)
    Documentation of my early work on STDP as a Caltech graduate student.

  • 1993-08-11
    Frustrated Synapse: Neural Network Training Algorithm (revised)

  • 1993-08-09
    Frustrated Synapse Lab Book
    I documented my early work on STDP using in a United States Air Force Academy (USAFA) journal.

  • 1993-05-06
    Frustrated Synapse: Neural Network Training Algorithm
    My earliest documentation on my STDP simulations. I worked on this while a First Lieutenant in the United States Air Force before I entered graduate school later that year at Caltech.

  • 1993 - 1997
    Synchronicity and Periodicity Research
    Research under the guidance of Dr. Bartlett Mel and Dr. Ernst Niebur on the theoretical analysis of a biologically realistic neuron model with dendritic tree using the simulation tool Neuron.

  • 1993 Oct - 1995 Jun
    M.Sc. Electrical Engineering
    California Institute of Technology, Pasadena, CA
    Focus: Computational Neuroscience, Neuromorphic VLSI

  • 1993 - 1997
    Spike-Timing Dependent Plasticity
    Invention of and research into a biologically plausible learning rule for spiking neurons.

  • 1990
    ART-1 Hardware and Software Implementation
    Design, implementation, and demonstration of the neural network ART-1 learning algorithm for pattern recognition. System included photodiode input, digital to analog conversion, RS-232 serial I/O circuitry, serial I/O software, software implementation of the ART-1 learning algorithm, and graphical output. Senior year independent study project based on my reading of Wasserman (1989) Neural Computing: Theory and Practice.

  • 1986 Jul - 1990 Jun
    B.Sc. Electrical Engineering
    United States Air Force Academy, Colorado Springs, CO
    Focus: Computer Engineering, Digital Electronics, Neural Networks


Software

  • 2011 - Present
    Event Related Neuronal Simulation Tool
    Updating Java-based CNS simulator for Johns Hopkins University.

  • 2011
    IntFire
    An animated interactive Java Swing applet simulating the Integrate-and-Fire spiking neuron model.

  • 2008-08-29
    Neuro
    Animated interactive Java Swing applet simulating the Hodgkin-Huxley neuron model.

  • 2008
    SIEVE
    Java OpenGL (JOGL) desktop application to demonstrate the software library for the Spike Interface Embodied Virtual Environment (SIEVE). The software rendered a 3D object and then decomposed it into red, green, and blue spike rasters simulating a virtual optic nerve. The viewpoint was controlled by keyboard events which were converted into effector spikes to simulate movement of the operator neuronal network in the virtual environment. This demonstration of the SIEVE software library did not provide the neuronal network operator; control was manual and the result was rendered to screen.

  • 2006 - 2007
    Infant
    Java Swing desktop application for newborn infant cognition experiments. The software presents visual stimuli and records responses triggered by a pacifier connected to an air pressure transducer via the game port.

  • 2005 - 2006
    Newt Cyborg
    Animated interactive Java Swing applet simulating a spike-driven neuroprosthetics interface.

  • 1996-09-07
    Dice
    Click the mouse and watch a simple neural net learn whether to flee or fight.

  • 1996-08-23
    Insight
    Goblins hunt kobolds in the dark using a neural network.

  • 1996
    BackpropXOR
    Animated interactive Java applet simulating the ANN backpropagation learning rule.

  • 1994-05-21
    HyperNet
    A computer simulation experiment to train a spiking neuronal network to learn a song using STDP. Written in the Ada programming language.

  • 1993 - 1995
    STDP source code
    STDP simulation source code in the Ada programming language.


Spike Timing Dependent Plasticity

I believe I was one of the first to research what is now known as spike timing dependent plasticity (STDP), having invented it in the Spring of 1993 as a biologically plausible neuronal network learning rule which I called Frustrated Synapse learning. My novel idea was to combine Hebbian learning with an additional rule that would make a synaptic weight more negative if it was triggered during the hyperpolarization of the target neuron. Through a number of computer simulation experiments that Summer, I observed that the ability of the algorithm to overcome the stability-plasticity dilemma and to stabilize fully recurrent networks.

In the Fall of 1993, I entered Caltech as a graduate student and threw myself into my simulations and course work in an effort to determine whether this was the learning rule used by real-life biological neurons. During that time, I was able to discuss the learning rule with many of my professors and classmates and increase the sophistication of my models, as demonstrated in some of my notes, student papers, and presentation slides from that period as listed below. When I left graduate school to enter industry in 1995, I was even more convinced that this learning algorithm was biologically plausible.

Upon returning to the field in 2003, I was pleasantly surprised to learn that this learning rule was demonstrated to exist in biological neurons through experiments performed by neuroscientists in 1997. What I have previously labeled Frustrated Inhibitive Synaptic Localized (FISL), Frustrated Synapse, Phase Covariance, or Hebbian Phase learning is now known as antisymmetric Hebbian or STDP.

I wrote my STDP simulations between 1993 and 1995 in the computer programming language Ada. The source code is archived online.


Synchronicity and Periodicity

Research under the guidance of Dr. Bartlett Mel and Dr. Ernst Niebur on the theoretical analysis of a biologically realistic neuron model with dendritic tree using the simulation tool Neuron. I worked on this primarily from 1993 to 1995 while I was a graduate student at Caltech and Dr. Mel and Dr. Niebur were post-docs in the Koch Lab. I continued working on this a bit more until around 1997. My main contribution to the paper was in running the Neuron simulations and in the discussions regarding the effects of synchronicity versus asynchronicity and periodicity versus aperiodicity in the synaptic inputs on the output firing rate.


Spike Interface Embodied Virtual Environment

Based on my previous work on the Neuroprosthetic Training Software, I conceived of the Spike Interface Embodied Virtual Environment (SIEVE) on 2007-01-03. The purpose of SIEVE was to create a virtual environment for simulated neuronal networks in which both effectors and sensors communicated via spikes. As in the Jointed Neuroprosthetic Trainer, spikes from peripheral nerves nerves would be used to control virtual representations of effectors such as limbs. In addition, SIEVE would also provide a simulated operator with feedback from a virtual sensor such as an optic nerve. The virtual optical nerve would be a neuromorphic spike raster representation of the 3D rendered scene from the viewpoint of the operator.

I worked on SIEVE as part of my doctoral studies. I wrote a Java OpenGL (JOGL) software library to convert 3D animation into separate visual streams of spikes as an interface to a neuronal network. I filed a provisional patent application on the software.

I submitted an Army SBIR proposal entitled Neuromimetic Flight Simulator in 2008 and an Army STTR proposal entitled Incremental Learning for Robot Sensing and Control in 2009.

I also played around with the LEGO Mindstorms NXT robot since it supports a Java-based programming language. My plan was to train the neuronal network in the SIEVE Virtual Reality Training Simulator using STDP and then validate using robotics.

I stopped working on SIEVE in 2009 when I discovered prior art in the research paper DiPaolo (2003) Evolving spike-timing-dependent plasticity for single-trial learning in robots. I then shifted my research focus to create a Computational Neuroethology (CNE) software library.

 
 
 
CroftSoft
 
 
About
Library
-Books
-Code
-Courses
-Links
-Media
-Software
-Tutorials
People
Portfolio
Update
 
 
Google
CroftSoft Web

 

Creative Commons License
© 2011 CroftSoft Inc.
You may copy this webpage under the terms of the
Creative Commons Attribution License.