Silicon Brains, Photonics, etc.

The semester is officially over, which is exciting: I finally get to get back into the swing of research (with the occasional GRE study break, of course). As such, blogging will tend to be lighter; but before I lock myself in the lab, here are a few things that came to pass while I was busy finishing up the semester…

Building Brains in Silicon
Among other things, I wrote a paper for my computational neuroscience class on – you guessed it – some really cool work coming out of Kwabena Boahen’s group (formerly here at Penn, now at Stanford) on silicon-based artifical neural systems. This is sometimes classed as ‘neuromorphic engineering’, a term (coined by Carver Mead in the 1980’s) which has come to refer to a relatively recent interdisciplinary paradigm dealing with the development and study of artificial neural systems, drawing on principles from such fields as physics, biology, and computer/electrical engineering to design electronic-based analogues of biological systems. A number of people are using this to try to design new VLSI-based systems based on biological systems.

Some others are trying to reverse this scenario: while ‘real’ neural systems are experimentally studied by neurobiologists while grossly simplified ones are modeled by computational neuroscientists, groups like Boahen’s are trying to bridge these modes of inquiry by exploiting similarities between electronic and neural circuits. Mahowald and Douglas wrote a seminal paper in 1991 describing the first ‘silicon neuron’, and a good deal of work has gone on since then. For example, a number of ‘thermodynamic’ models of ion channels have been developed, building on concepts like Hodgkin/Huxley-type models. Anyway, by exploiting the beautiful similarity between ion channels and metal-oxide-semiconductor (MOS) transistors as two-state systems (simplistically, ion channels are either open or closed, with the energy barrier – and hence the transition rate – between the two states being modulated via, for example, a voltage; on the other hand, a voltage applied across the source and the drain of a MOSFET causes charges to diffuse through the ‘conduction channel’, with the effective barrier to this diffusion being modulated by a gate voltage), Boahen and his graduate student Kai Hynna have recently taken an important step toward ‘building a brain in silicon’. Using an approach that combines the advantages of experiment and artificial modeling, they have developed a simple electronic circuit that replicates the nonlinear dynamics of the gating particles of voltage-dependent ion channels.

References:
- Hynna and Boahen’s recent paper: K. M. Hynna and K. Boahen, Neural Computation 19, 327 (2007).
- 1991 silicon neuron paper: M. Mahowald and R. Douglas, Nature 354, 515 (1991).
- Thermodynamic models of ion channels: A. Destexhe and J. R. Huguenard, J. Comput. Neurosci. 9, 259 (2000).

Update: I guess Tech Review thought this stuff is cool, too: the latest issue has an article on Boahen’s work. It takes a broader view of his work than I have above - I just focused on one particular aspect.

Quasicrystals and Complex Materials as 3D Photonic Structures
I wrote another paper for my modern optics class, based on this recent experimental paper by Man, Megens, Steinhardt and Chaikin on three-dimensional quasicrystals as complete photonic bandgap materials. Here’s the deal: since Schrödinger’s wave equation and the electromagnetic wave equation are formally similar (neglecting spin statistics), it isn’t all that surprising that a number of analogies exist between electronic waves and light. In particular, electromagnetic waves can propagate in structures of periodic dielectric constant, and interference due to multiple Bragg reflections from these interfaces leads to directional-dependent energy band gaps. A major goal is to try to develop artificial structures to act as complete, omnidirectional photonic bandgap (PBG) crystals with bandgaps in the visible regime (wavelength ~ 400-700nm), and a lot of effort has gone into this. Interestingly, recent innovations in materials science and the study of complex materials – such as quasicrystals (QC), liquid crystals (LCs), and colloidal self-assembly – have breathed new life into this quest.

References:
- Experimental confirmation of the almost-spherical effective Brillouin zone (and hence the potential of developing a 3D PBG structure) of a macroscopic 3D icosahedral photonic QC: W. Man, M. Megens, P. J. Steinhardt and P. M. Chaikin, Nature 436, 993 (2005).
- Experimental approach towards assembling 3D analogues of the QC structures studied by Man et al. on a smaller scale using holographic optical trapping: Y. Roichman and D. G. Grier, Opt. Exp. 13, 5434 (2005).
- Another experimental approach, using a novel 7-beam optical interference holography technique: W. Y. Tam, Appl. Phys. Lett. 89, 251111 (2006).
- Using nematic liquid crystals in ‘inverse opal’ structures as PBG materials (tuned by parameters such as an external electric field) – for example, since liquid crystals are birefringent, modulating their orientational order using a field can influence their optical properties (a principle on which liquid crystal displays are based): K. Busch and S. John, Phys. Rev. Lett. 83, 967 (1998).
- Recent computational work has indicated a feasible method of fabricating 3D visible PBG crystals with two different types of lattice structure using self-assembly of a mixture of colloidal spheres of two different sizes: A. P. Hynninen, J. H. Thijssen, E. C. Vermolen, M. Dijkstra, and A. van Blaaderen, Nature Mater. 6, 202 (2007).

Talks Part 1: Modeling Cells

I always enjoy attending good talks, and we’ve had quite a few lately: they’ve either been relevant to my current research, interesting applications of concepts I’ve encountered before, or just plain cool. (And of course, going to talks - or blogging, for that matter - is always a good way of taking a break from working all the time i.e. staying sane.) The material presented is always a good springboard for learning more. Here are a few summaries and references for the talks that stuck out the most, mainly because they dealt with things that I hadn’t directly encountered before.

Modelling Cell Motion and Morphogenesis: Mark Alber (Notre Dame)
Although this was billed as an applied math talk, it felt more like a physics talk: the speaker focused on trying to convey what was actually going on and less on the mathematical details involved in modeling it, which was refreshing. I’ve been to too many math talks where the speaker gets hung up on mathematical details, and the big picture somehow gets left out. Anyway, Prof. Alber is an applied mathematician at Notre Dame who spends a good deal of time modeling cells and biological processes at various scales (hence the term multiscale modeling) using ideas from statistical mechanics. The point is this: different modeling methods have their advantages and disadvantages. For example, macroscopic continuum methods abstract away many (often crucial) things and can miss different kinds of phenomena, while microscopic cell-level models - in which stochasticity is very important - can be computationally very intensive. This is particularly important in biological systems, where important processes take place at pretty much all scales; ideally, one would be able to construct a 3-dimensional model of a system and be able to zoom in and out with ease. This is what Alber et al. have been working on, in two different ways.

The first method is a 3D stochastic model of myxobacteria dynamics based on a lattice-gas cellular automata model, and using this, they’re able to study experimentally-observed phenomena like rippling, the formation of things like ’streams’ and ‘traffic jams’, and cell swarming/aggregation. The second method treats the cells as extended objects and goes off the philosophy that “while individual organisms and organs have very different structures and behaviors, many of the underlying interactions and components are the same.” In particular, it is an implementation of the Cellular Potts Model (CPM) of statistical mechanics, a non-equilibrium variant of the Ising model, coupling this to a continuum reaction-diffusion model for morphogen production/diffusion and a set of conditions dictating how genes are regulated. In particular, each cell is represented as a cluster of pixels in the CPM (with a multidimensional index indicating the type of cell) and interacts with other cells via a pairwise adhesion, and the cool thing is that they can use this to model - to a certain extent - limb formation in things like growing chicken embryos.

Further reading…
- Modeling myxobacteria dynamics: O. Sozinova et al., “A Three-Dimensional Model of Myxobacterial Aggregation by Contact-mediated Interactions“, PNAS 102 11308 (2005) and D. Kaiser, “Coupling Cell Movement to Multicellular Development in Myxobacteria“, Nature Reviews Microbiol. 1 45 (2003).
- Modeling limb formation: R. Chaturvedi et al., “On Multiscale Approaches to 3-Dimensional Modeling of Morphogenesis“, J. R. Soc. Interface 2 237 (2005).
- Somewhat related: D. A. Beysens et al., “Cell Sorting is Analogous to Phase Ordering in Fluids“, PNAS 97 9467 (2000) and W. Zeng et al., “Non-Turing Stripes and Spots: a Novel Mechanism for Biological Cell Clustering“, Physica A 341 482 (2004). In particular, I find the similarities between figure 2 of the latter reference and this picture of stripe formation in ferrofluids in one of my previous posts on electronic liquid crystals are striking.

Neural Networks III

Last week I wrote the second post in my ongoing series of posts on Computational Neuroscience, based on a class I’m taking on the subject. I take notes on my computer in class anyway, so what I post here is a modified version of those and hence a reasonably good course chronicle.

While a lot of the development can be very technical, I’m trying to make these posts accessible to most people with a general interest in biophysics, neural networks or computational neuroscience. This means that I have to gloss over a lot of interesting details, including a lot of fun derivations, but that’s ok - for one thing, I don’t know how to easily incorporate equations into these posts (nor do I have the time to). However, I do provide references to more detailed papers when I feel the need.

Last week’s subject was Mikhail Bongard’s problems of pattern recognition, Herrnstein’s discovery that pigeons were particularly adept at solving them, more perceptrons and the credit assignment problem, and how backpropagation provided a means of solving it. The week before, I introduced McCulloch-Pitts neurons and perceptrons, particularly the perceptron convergence theorem and linearly separable problems. This week builds on the past two weeks: we’ll be looking at NETtalk, the Ising Model and Hopfield Networks, and Attractor Dynamics.

Neural Networks II

Last week’s subject was McCulloch-Pitts neurons and perceptrons, particularly the Perceptron Convergence Theorem and linearly separable problems. This week builds on that, and this post’s subject is Pigeons, the Credit Assignment Problem, and Backpropagation.

Neural Networks I

Yesterday was my first computational neuroscience class, a field that is totally foreign (and interesting) to me. We started off talking about neural networks, a field which has its roots in systems theory and the work of Nicholas Rashevsky, “a Russian émigré theoretical physicist who developed a program in ‘mathematical biophysics’ at the University of Chicago during the 1930s”. For example, he was one of the first scientists to analyze traffic mathematically (which confuses me a bit, since I’m not sure how much of an issue traffic was in the ’20’s or ’30’s).

At any rate, work building on that of Rashevsky and others plodded along for a while, until the seminal work of Warren McCulloch and Walter Pitts at MIT (A Logical Calculus of Ideas Immanent in Nervous Activity, 1943, Bulletin of Mathematical Biophysics 5:115-133) while considering the question: what kind of things can be computed mathematically? Today the subject of their paper is often referred to as a McCulloch-Pitts neuron.

A McCulloch-Pitts neuron is quite simple: essentially, it is a simplified tunable linear threshold unit (the Wikipedia article I’ve linked to does a particularly good job at explaining it), and what McCulloch and Pitts showed was that with a network of these, one can compute any logical expression, or any combination of logical predicates. The device is tunable in the sense that each input can be weighted (and a tunable ‘bias’ is often applied, too), and by varying the different weights with respect to a given threshold, the operation being computed can either be AND, OR, NOT, etc. Clearly this is incredibly simplified - no probabilities, no further considerations in the mechanism, nothing but the bare bones. The fact that a network of these can be so powerful was a big deal in the ’40’s, and people began wondering whether such ‘neural networks’ could be made to act like a brain. Clearly the jump from simple logical operations using a network of McCulloch-Pitts neuron to advanced concepts (like pattern recognition) is huge.

Twenty years later, Frank Rosenblatt - an engineer at Cornell - came up with what he called the Perceptron. Essentially, Rosenblatt was looking for a way to come up with an image recognition machine, equipped with some programmatic behavior to make it better as time went on. This resembles neural networks to a large extent - here’s a picture from Wolfram research:

As in McCulloch-Pitts neural networks, the weighted inputs (and bias, which in the image is 1) are summed, and assessed against a given threshold. Each input comes from a set of ‘feelers’ on the screen where the image is displayed e.g. photoreceptors. Things get a teensy bit complicated, and I’m trying to keep this equation-free, so I’ll jump to the punchline: Rosenblatt was able to show that for a linearly separable problem (that is, trying to distinguish between two different things that form different ‘clumps’ in an n-dimensional parameter space that can be separated by an n-1 dimensional hyperplane), the Perceptron only takes a finite number of examples for it to learn successfully. This Perceptron Convergence Theorem was again a Very Big Deal, and led to a good deal of research in the field for a little under 10 years…

Until it all stopped. Why? Because of a monograph (Perceptrons) - published by Minsky and Papert (at MIT as well) - that put an end to it all, and gave rise to the New and Improved (i.e. ‘hot’) field of Artificial Intelligence. For many years after that, the field of neural networks was completely and utterly dead.

Until next time…