IJCNN05 Montreal Workshop, 19:00-22:00 Thursday 04Aug05 NNs, Bio- and Neuro- Informatics Junk DNA and Neural Networks: conjecture on directions and implications Mr. Bill Howell [email protected](use Menu selection “View->Notes” to see the notes which accompany many of the slides in this MS Powerpoint presentation)
33
Embed
IJCNN05 Montreal Workshop, 19:00-22:00 Thursday 04Aug05 NNs, Bio- and Neuro- Informatics Junk DNA and Neural Networks: conjecture on directions and implications.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
IJCNN05 Montreal Workshop, 19:00-22:00 Thursday 04Aug05
NNs, Bio- and Neuro- Informatics
Junk DNA and Neural Networks:conjecture on directions and implications
Mr. Bill Howell [email protected](use Menu selection “View->Notes” to see the notes which accompany many of the
slides in this MS Powerpoint presentation)
The views expressed in this presentation are personal and speculative. They are in no way related to the research, policies, viewpoints, and programs of my current employer, the federal government department “Natural Resources Canada”.
To the best of my knowledge, there is no work underway or planned in this area within the federal government at this time.
Bill Howell
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Outline
1.Revolution beyond the "central dogma of biology" – DNA: function beyond gene-protein through “junk”– Junk = non-protein coding, incl non-exon, epigenetics, other?
2. Related trends with NN architectures and processes• Structure, function, messaging, dynamic transitions• Learning, control, planning, behaviour, goals
3. Computational Neuro-Genetic Modelling (CNGM)1. Bio- and Neuro- informatics relevance2. Expanded approach to ANNs
4. Implications for the brain and the mind● Inheritance of vast knowledge, grammar (not just linguistic!)1. Multiple parallel “behaviours & personalities” (computing)2. Evolutionary theory3. Mindcode (highly speculative)
1. Revolution beyond the "central dogma of biology"
2. Related NN trends with architectures and processes
3. Computational Neuro-Genetic Modelling
4. Implications for the brain and the mind
“The gold standard for NNs, far off in the distance, IS the human brain...”
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Junk DNA as code
● A revolution in the "central dogma of biology"?– John Mattick (UQueensland), others over several years
● Eukaryotic DNA coding >>> genes for proteins • ~1.5% of human DNA codes for proteins, but most DNA
transcribed to RNA!● poor relation between organism's complexity & # of protein-
coding genes, more consistent relation with non-coding DNA?● DNA expression
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Junk DNA as code (cont'd)
– Genes: conventionally thought as literally "assembly language programming" for proteins – perhaps the simplest and lowest level of programming on the DNA?
– Junk -> RNA and micro-RNA, regulatory role● Chromatin – mask/reveals DNA code● Architecture – need precise plans, “drawings” - highly
specific
– Cambrian period bio-complexity explosion (~1 Gy ago)
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
John Mattick: “Cambrian complexity explosion”
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Possible Examples?● Timing circuits in cells/ organism (special emphasis on time,
sequencing, coordination of parallel and sequential processes)– multitemporal, multifractals
• High cross-over rate genes (individual variabilities such as appearances)
• Linguistics – Chomsky, Pinker (1994), note that the distinction between genes and "non-gene" DNA is not emphasized by many authors
● Feedforward - basis of current descriptions of npcRNA
● Feedback from neuron to DNA– A given for regulatory control, but is this limited to
“normal” physiological sensing/ control pathways?– Is there a means for the neurons to direct SPECIFIC
DNA expression (protein or non-protein coding DNA), in a way that isn’t simply physiology, but is directly for specific information processing/ data tasks?
– Could lead to Recurrent Neural Networks (RNNs) that use DNA coding/ “programs” (programming metaphor)- with very powerful advantages!
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Neuron to DNA signal & control?
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Advantages of Junk DNA as Code● Mattick:
– away from the combinatorial soup of proteins for regulation, towards a highly specific "program" to direct spliceosomes (low side effects, fast, efficient)
– specification of architecture/ growth allows vastly greater complexity of organisms (just what we are looking for with the brain!)
– alternative splicing & "overloading" of genes – assemble protein-coding-RNA in one of several ways, code has different functionality in different cell types
● Other possible advantages - metaphor of computing– Vastly parallel (as fits NNs) – Not just “static” code – dynamic interactions between coding– ??Co-resident junk & protein code (data/ methods -> super-
objects), “calls” to ensembles of NNs
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Challenges – extrapolating to the brain● Genes – "relatively easy" to see, identify coding
it's still a work in progress, start/ stop sequences, intron removal etc, how to decide when two conformations are possible?
● Cell physiology -> "event" amoung many others happening in parallel, still can "see" amongst all of the mRNA & cell signalling happening at the time
● Ontogeny (growth) - visibility like cell physiology, but how easy is it to quantify subtle structural changes?
● But what about more abstract processes, and thought? How can one identify these?how to link code & its effect?
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
What toolsets do geneticists have & need?
● Already a great base & fallback position – current genomics/ bioinformatics/ neuroinformatics
● Need - greater exactness of:– [Reading, precisely changing] DNA code, RNA & epigenetics– Measuring structural changes in neurons & NN ensembles– Also, what are “feedback” mechanisms from NN changes to
DNA?
● Signaling – how good is this for the purposes in mind?
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Learning &Training (cont'd)
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
"Small-world” universal function approximation
● Small world domain – what is the smallest set of NNs, of various functional capabilities (general to specific), that is sufficient to solve most of the problems in a domain of interest?
● How does one restructure/ combine/ train these NN modules to obtain ultra-fast, accurate learning/ control? (eg for control – ObjectNet adaptive critics)
● What happens for a much broader or universal domain? (combine both general & special NN)
● Building from level of abstraction to the next● Converse issue – false confidence in good fits (eg Global
Circulation Models for climate, Valdes @ IJCNN05)
3. Computational Neuro-Genetic Modelling
4. Implications for the brain and the mind
1. Revolution beyond the “central dogma of biology"
2. Related NN trends with architectures and processes
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Expanded approach to NNs
● Nik Kasabov, Benuskova, Wysoski (Aukland UTech) - architectures for gene networks, extending the concept to advance NN architectures
● What are the desired capabilities/ opportunities?– Co-resident [code, genes] - beyond OOPS?, new
RNNs?– Data [delivery mechanism] - code segments identify
data (DNA or RNA code keys physically bring data and destination together!), data can drive code & architecture…
– Dynamic structures – switch/evolve in real time! Approximate Dynamic Programming (ADP)
● Hypothesis of bidirectional action:– Junk DNA -> drives neuron states– Neuron states -> initiate junk DNA sequences
● Different object inputs to, and functional behaviour of, a single NN module or architecture of NN modules
● Rapid reconfiguration of ensembles of NN modules along “high likelihood” arrangements, longer term more exhaustive evolution
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Expanded NNs (cont'd)
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
What differentiates CNGMs from current NNs?
● Will they simply be faster/ more accurate?– Like fixed weight NN - much faster system
identification/ learning
● Or are there new capabilities that will arise?– Greater “aptitude” for symbolism?
● Semantics and Logic as emergent properties for very complex systems (abstraction -> soft logic, giving explainability)
– Greater functional/ mapping specificity– Ease of combining networks of NN modules?
● What else? (my feeling: HUGE conceptual advances)
4. Implications for the brain and the mind
1. Revolution beyond the “central dogma of biology"
2. Related NN trends with architectures and processes
3. Computational Neuro-Genetic Modelling
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Implications - for the brain and the mind
● Inheritance of vast knowledge (content) - form very specific to general and to highly abstract– Data, procedures, “operating systems”, behaviours …– Selection of the right blends of different levels of
abstraction for inheritance– Eg language - could “know” all words, but it is the
dynamic bindings that give language its power– Far beyond “Nature versus Nurture” (which was
somewhat a ?dysfunctional? discussion anyways - eg ontogeny effects on identical twins, compression effects in relating synapses to inherited DNA). The problem with dichotomies…
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Implications (cont’d)● Inheritance of a vast grammar
– As per Chomsky, but not just linguistics - much more fundamental than that, at the connectionist levels as well as at the symbolic/ linguistic levels
● Multiple parallel behaviours & personalities– But will this somehow affect goal identification/ prioritization?
● Problem decomposition/ modularisation, reconstruction– Multiple models that are dissimilar, yet collaborate effectively
and simultaneously on a variety of models of the problem(s) being addressed
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Evolutionary Theory● Lamarckian heredity - pass on capabilities (strong arms,
● Would a Lamarckian evolutionary approach “re-discover” basic principles and laws that have long been established in other areas?– learning (pass on through teaching, apprenticeship,
experience, management fashions)– Economics, organization structure and processes?? etc
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Mindcode● "Given that computer code is used to program
computers, then mindcode..."– The perspective here isn't to "program" a child/adult brain by
some external means, but rather to seek an understanding of junk DNA coding (and perhaps other sources of coding such as epigenetics) that may define the basis of our brains from conception. What might such code tell us about ourselves and our history that is different from current psychology, sociology, anthropology, management theory, economics?
● This is pure speculation and fantasy, but I think that it is a useful fantasy to drive lines of investigation, to maintain an awareness of the types of results that we should be looking for (its easier to find something when you are looking for it).
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
Ast, Gil “The alternative genome” Scientific American, vol 292 iss 4, Apr05, pp58-65
· Doja, Kenji; Dayan, Peter; Hasselmo, Michael (guest editors) "2002 Special Issue: Computational Models of Neuromodulation" Neural Networks, Vol 15 Nos 4-6, June-July 2002: Kenji Doya "Metalearning and neuromodulation" pp495-506
· Eddy, Sean R. “Non-coding RNA genes and the modern RNA world” Nature Reviews Genetics, vol 2, pp 919-929, December 2001(from Gibbs)
Gibbs, W. Wayte "The unseen genome: Gems among the junk" Scientific American, vol 289 no 5, Nov03 pp46-53
Ingoglia, N. organizer (American Society of Neurochemistry (ASN)) “RNA Interference (small RNAs): Applications to neural systems” Pre-meeting workshop, ASN Annual Meeting, 14-18Aug04, New York city
· Krichevsky, Anna (Harvard Medical School) - research focussed on neurons: Kim, J; Krichevsky A; Grad, Y: Hayes, G.D; Kosik, K.s; Church, G.M; Ruvkun, G “Identification of many microRNAs that copurify with polyribosomes in mamalian neurons” PNAS vol 101 no 1, 06Jan04, pp360-365
Mattick, John S. “Challenging the dogma: the hidden layer of non-protein-coding RNAs in complex organisms” BioEssays, vol 25 pp930-939, Oct03
· Mattick, John S. (UQueensland) "The hidden genetic program of complex organisms" Scientific American, Oct04 pp60-67. See http://imbuq.edu.au/groups/mattick
2. Related trends with NN architectures and processesa) Guang-Bin Huang, Qin-Yu Zhiu, Chee-Kheong Siew, Nanyang TechU, Singapore
"Extreme learning machine: A new learning scheme of feedforward neural networks" IJCNN04 Budapest
d) ?"Co-Evolutionary Learning of Liquid Architectures" Igal Raichelgauz, Karina Odinaev Yehoshua Y. Zeevi, Israel unpublished yet?
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
References
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
References
3. Computational Neuro-Genetic Modellinga) N. Kasabov, L. Benuskova, S.G. Wysoski (Knowledge Engng & Discovery Research Institute
(Kedri), Aukland UofTech, New Zealand) "Computational neurogenetic modelling: Gene networks within neural networks" IJCNN04 Budapest
b) Rui Xu and Donald C. Wunsch v- see IJCNN05 references
International Joint Conference on Neural Networks 2005, Montreal NNs and Bio-/Neuro- Informatics
IJCNN05 References IJCNN05 = International Joint Conference on Neural Networks 2005, Montreal, International
Neural Netwsork Society and IEEE Computational Intelligence Society. Because of the ease of accessing conference papers, many are listed below. This is certainly NOT exhaustive!!
a) Relevant sessions include:– S2 "Computational Neuro-Genetic Modelling" chair Nik Kasabov– S8 "Constructive/Hierarchical Self-Organizing Maps" chairs Ernesto Cuadros-Vargas and
Roseli Francelin Romero– Sa "Computational Dynamical Modeling with Echo State Networks" chairs Yadunandana Rao
and Jose Principe– Sf "Evolvable and Emergent Neural Systems" chairs Seong Kong and Jacek Zurada– P1-Gf "Neural network architectures and structures" chairs: IJCNN05 program chairs
b) Related papers include:1. 1728 "Gene Regulatory Networks Inference with Recurrent Neural Network Models" Rui Xu
and Donald C. Wunsch II, ACIL, University of Missouri-Rolla, United States2. 1016 "Functional Grouping of Genes Using Spectral Clustering and Gene Ontology" Nora
Speer, Holger Froehlich, Christian Spieth and Andreas Zell, Centre for Bioinformatics Tuebingen (ZBIT), Germany
3. 1603 "Modeling Cortico-Subcortical Interactions During Planning, Learning, and Voluntary Control of Actions" Daniel Bullock, Boston University, United States
4. 1257 "Protein Sequence Classification Using Extreme Learning Machine" Dianhui Wang, La Trobe University, Australia; and Guang-Bin Huang, Nanyang TechUniversity, Singapore