hinton

Curriculum Vitae
Georey E. Hinton
January 8, 2014
Citizenship: Canadian (also British)
Address: Department of Computer Science
University of Toronto
10 Kings College Road
Toronto, Ontario, M5S 3G4
Canada
Phone: 416-978-7564
Email: hinton@cs.toronto.edu
Higher Education and Qualications
1967 - 1970 Cambridge University, B.A. Hons (Experimental Psychology)
1972 - 1975 Edinburgh University, PhD. in Articial Intelligence (awarded 1978)
Professional Experience
Jan 76 - Sept 78 Research Fellow
Cognitive Studies Program, Sussex University, England
Oct 78 - Sept 80 Visiting Scholar
Program in Cognitive Science, University of California, San Diego
Oct 80 - Sept 82 Scientic Ocer
MRC Applied Psychology Unit, Cambridge, England
Jan 82 - June 82 Visiting Assistant Professor
Psychology Department, University of California, San Diego
Oct 82 - June 87 Assistant Professor then Associate Professor
Computer Science Department, Carnegie-Mellon University, Pittsburgh, USA
Jul 87 - June 98 Professor
Computer Science Department, University of Toronto, Canada
Jul 98 - Sep 01 Founding Director of the Gatsby Computational Neuroscience Unit
University College London, England
Oct 01 - Professor
Computer Science Department, University of Toronto, Canada
(Promoted to \University Professor" in 2006)
(became part-time in Mar 2013).
Mar 13 - Distinguished Researcher, Google Inc. (part-time).
1
Professional Recognition
Fellowships
2003 Honorary Foreign Member of the American Academy of Arts and Sciences
2003 Fellow of the Cognitive Science Society
1998 Fellow of the Royal Society
1996 Fellow of the Royal Society of Canada
1991 Fellow, Association for the Advancement of Articial Intelligence
1987 Fellow, Canadian Institute for Advanced Research (1987-1998; 2004-2014)
Awards and Honorary Degrees
2013 Doctorat honorique, University of Sherbrooke
2012 Killam Prize in Engineering
2010 Gerhard Herzberg Canada Gold Medal
2011 Honorary Degree of Doctor of Science, University of Sussex
2005 IJCAI Research Excellence Award
2001 Honorary Degree of Doctor of Science, University of Edinburgh
2001 The David E. Rumelhart Prize
1998 IEEE Neural Networks Pioneer Award
1992 ITAC/NSERC award for academic excellence.
1990 IEEE Signal Processing Society Senior Award
Named Lectures
2012: Killam Prize Lecture, McGill
2011: The Foundation Lecture, Royal Canadian Institute
2010: The Hans-Lukas Teuber Lecture, MIT
2010: The \Big Thinkers" lecture, Yahoo, San Jose
2010: The Rockwood Memorial Lecture, UC San Diego
2009: The Ed Posner Lecture, NIPS-09, Vancouver
2009: The Ian Howard Lecture, York University
2006: The Graham Lecture, University of Toronto
2003: The Pinkel Lecture, University of Pennsylvania
2001: The David E. Rumelhart Prize Lecture, Edinburgh
1998: The Rockwood Memorial Lecture, UC San Diego
1995: The Rockwood Memorial Lecture, UC San Diego
1993: The Herzberg Lecture, Ottawa.
1993: The Broadbent Lecture, London, UK.
1992: The Benjamin Meaker Lectures, Bristol University (5 lectures).
1991: The St Andrews Easter Lectures, St Andrews (6 lectures).
1989: The fourth annual Hebb lecture, Dalhousie University, Halifax.
1989: The Sun Annual Lectures, University of Manchester (8 lectures)
1987: The Weigand Lecture, University of Toronto
1986: The David Marr Memorial Lecture, Kings College Cambridge
2
PUBLICATIONS
Refereed Journal Papers
1. Ranzato, M., Mnih, V., Susskind, J. and Hinton, G. E. (2013) Modeling Natural Images Using Gated
MRFs. IEEE Trans. Pattern Analysis and Machine Intelligence, 35:9, pp 2206-2222.
2. Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen,
P., Sainath, T., and Kingsbury, B. (2012) Deep Neural Networks for Acoustic Modeling in Speech
Recognition. IEEE Signal Processing Magazine, 29:6, pp 82-97.
3. Salakhutdinov, R. R. and Hinton, G. E. (2012) An Ecient Learning Procedure for Deep Boltzmann
Machines. Neural Computation, 24, pp 1967-2006.
4. Hinton, G. E. (2013) Where do features come from? Cognitive Science, pp 1-24.
5. van der Maaten, L. J. P. and Hinton, G. E. (2012) Visualizing Non-Metric Similarities in Multiple
Maps. Machine Learning, 87, pp 33-55.
6. Mohamed, A., Dahl, G. and Hinton, G. E. (2012) Acoustic Modeling using Deep Belief Networks.
IEEE Transactions on Audio, Speech, and Language Processing, 20, pp 14-22.
7. Taylor, G. W, Hinton, G. E., and Roweis, S. (2011) Two distributed-state models for generating high-
dimensional time series Journal of Machine Learning Research, 12, pp 863-907.
8. Hinton, G. E. and Salakhutdinov, R. (2011) Discovering Binary Codes for Fast Document Retrieval by
Learning Deep Generative Models. Topics in Cognitive Science, 3:1, pp 74-91.
9. Schmah, T., Yourganov, G., Zemel, R. S., Hinton, G. E., Small, S. l., and Strother, S. C. (2010)
Comparing Classication Methods for Longitudinal fMRI Studies Neural Computation, 22, pp 2729-
2762.
10. Memisevic, R. and Hinton, G. E. (2010) Learning to represent spatial transformations with factored
higher-order Boltzmann machines. Neural Computation, 22, pp 1473-1492.
11. Hinton, G. E. (2010) Learning to represent visual input. Philosophical Transactions of the Royal
Society, B. 365, pp 177-184.
12. Sutskever, I. and Hinton, G. E. (2010) Temporal Kernel Recurrent Neural Networks Neural Networks,
23, pp 239-243
13. Salakhutdinov, R. and Hinton, G. E. (2009) Semantic Hashing. International Journal of Approximate
Reasoning, 50, pp 969-978.
14. Mnih, A., Yuecheng, Z., and Hinton, G. E. (2009) Improving a statistical language model through
non-linear prediction. NeuroComputing, 72, pp 1414-1418.
15. van der Maaten, L. J. P. and Hinton, G. E. (2008) Visualizing Data using t-SNE. Journal of Machine
Learning Research, 9(Nov) pp 2579-2605.
16. Sutskever, I. and Hinton, G. E. (2008) Deep Narrow Sigmoid Belief Networks are Universal Approxi-
mators Neural Computation, 20, pp 2629-2636.
17. Hinton, G. E. (2007) Learning multiple layers of representation. Trends in Cognitive Science, 11, pp
428-434.
18. Hinton, G. E. and Salakhutdinov, R. (2006) Non-linear dimensionality reduction using neural networks.
Science, 313, pp 504-507, July 28 2006.
9. Hinton, G. E., Osindero, S., Welling, M. and Teh, Y. (2006) Unsupervised discovery of non-linear
structure using contrastive back-propagation. Cognitive Science, 30, (4), pp 725-731.
20. Hinton, G. E., Osindero, S. and Teh, Y. (2006) A fast learning algorithm for deep belief nets. Neural
Computation, 18, pp 1527-1554.
21. Osindero, S., Welling, M. and Hinton G. E. (2006) Topographic Product Models Applied To Natural
Scene Statistics. Neural Computation, 18, pp 381-414.
22. Memisevic, R. and Hinton, G. E. (2005) Improving dimensionality reduction with spectral gradient
descent. Neural Networks, 18, pp 702-710.
23. Sallans, B and Hinton, G. E. (2004) Reinforcement Learning with Factored States and Actions. Journal
of Machine Learning Research, 5 pp 1063{1088.
24. Welling, M., Zemel, R. and Hinton, G. E. (2004) Probabilistic sequential independent components
analysis. IEEE Transactions on Neural Networks, 15, pp 838-849.
25. Teh, Y. W, Welling, M., Osindero, S. and Hinton G. E. (2003) Energy-Based Models for Sparse
Overcomplete Representations. Journal of Machine Learning Research, 4, pp 1235-1260.
26. Friston, K.J., Penny, W., Phillips, C., Kiebel, S., Hinton, G. E., and Ashburner, J. (2002) Classical
and Bayesian Inference in Neuroimaging: Theory. NeuroImage, 16, pp 465-483.
27. Hinton, G. E.(2002) Training Products of Experts by Minimizing Contrastive Divergence. Neural
Computation, 14, pp 1771-1800.
28. Mayraz, G. and Hinton, G. E. (2001) Recognizing hand-written digits using hierarchical products of
experts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, pp 189-197.
29. Paccanaro, A., and Hinton, G. E. (2000) Learning distributed representations of concepts from rela-
tional data using linear relational embedding. IEEE Transactions on Knowledge and Data Engineering,
13, 232-245.
30. Ueda, N. Nakano, R., Ghahramani, Z. and Hinton, G. E. (2000) SMEM Algorithm for Mixture Models.
Neural Computation, 12, 2109-2128.
31. Ghahramani, Z. and Hinton, G.E. (2000) Variational Learning for Switching State-space Models. Neu-
ral Computation, 12, 831-864.
32. Ueda, N. Nakano, R., Ghahramani, Z. and Hinton, G. E. (1999) Split and Merge EM Algorithm
for Improving Gaussian Mixture Density Estimates. Journal of VLSI Signal Processing Systems, 26,
133-140.
33. Frey, B. J., and Hinton, G. E. (1999) Variational Learning in Non-linear Gaussian Belief Networks.
Neural Computation, 11, 193-214.
34. Ennis M, Hinton G, Naylor D, Revow M, Tibshirani R. (1998) A comparison of statistical learning
methods on the GUSTO database. Statistics in Medicine, 17 2501-2508.
35. Tibshirani, R. and Hinton, G.E. (1998) Coaching variables for regression and classication. Statistics
and Computing, 8, 25-33.
36. de Sa, V. R. and Hinton, G. E. (1998) Cascaded Redundancy Reduction. Network: Computation in
Neural Systems, 9, 73-84.
37. Fels, S. S. and Hinton, G. E. (1997) Glove-TalkII: A neural network interface which maps gestures to
parallel formant speech synthesizer controls. IEEE Transactions on Neural Networks, 8, 977-984.
38. Hinton, G. E. and Ghahramani, Z. (1997) Generative Models for Discovering Sparse Distributed Rep-
resentations. Philosophical Transactions of the Royal Society, B. 352, 1177-1190.
4
39. Frey, B. J., and Hinton, G. E. (1997) Ecient stochastic source coding and an Application to a Bayesian
Network Source Model. The Computer Journal, 40 (2).
40. Hinton, G. E., Dayan, P. and Revow M. (1997) Modeling the manifold of images of handwritten digits.
IEEE Transactions on Neural Networks, 8, 65-74.
41. Williams, C. K. I., Revow, M. and Hinton, G. E. (1997) Instantiating deformable models with a neural
net. Computer Vision and Image Understanding. 68, 120-126
42. Dayan, P. and Hinton, G. E. (1997) Using Expectation-Maximization for Reinforcement Learning.
Neural Computation, 9, 271-278.
43. Oore, S., Hinton, G. E. and Dudek, G. (1997) A mobile robot that learns its place. Neural Computation,
9, 683-699.
44. Dayan, P. and Hinton, G. E. (1996) Varieties of Helmholtz Machine. Neural Networks, 9, 1385-1403.
45. Revow, M., Williams, C. K. I. and Hinton, G. E. (1996) Using Generative Models for Handwritten
Digit Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18, 592-606.
46. Dayan, P., Hinton, G. E., Neal, R., and Zemel, R. S. (1995) Helmholtz Machines. Neural Computation,
bf 7, 1022-1037.
47. Hinton, G. E., Dayan, P., Frey, B. J. and Neal, R. (1995) The wake-sleep algorithm for self-organizing
neural networks. Science, 268, pp 1158-1161.
48. Zemel, R. S. and Hinton, G. E. (1995) Learning Population Codes by Minimizing Description Length
Neural Computation, 7, 549-564.
49. Becker, S. and Hinton, G. E. (1993) Learning mixture models of spatial coherence. Neural Computation,
5, 267-277.
50. Nowlan. S. J. and Hinton, G. E. (1993) A soft decision-directed LMS algorithm for blind equalization.
IEEE Transactions on Communications, 41, 275-279.
51. Fels, S. S. and Hinton, G. E. (1992) Glove-Talk: A neural network interface between a data-glove and
a speech synthesizer. IEEE Transactions on Neural Networks, 3.
52. Becker, S. and Hinton, G. E. (1992) A self-organizing neural network that discovers surfaces in random-
dot stereograms. Nature, 355:6356, 161-163.
53. Nowlan. S. J. and Hinton, G. E. (1992) Simplifying neural networks by soft weight sharing. Neural
Computation, 4, 173-193.
54. Jacobs, R., Jordan, M. I., Nowlan. S. J. and Hinton, G. E. (1991) Adaptive mixtures of local experts.
Neural Computation, 3, 79-87.
55. Hinton, G. E. and Shallice, T. (1991) Lesioning an attractor network: Investigations of acquired
dyslexia. Psychological Review 98, 74-95.
56. Hinton, G. E. (1990) Mapping part-whole hierarchies into connectionist networks. Articial Intelli-
gence, 46, 47-75.
57. Hinton, G. E. and Nowlan, S. J. (1990) The bootstrap Widrow-Ho rule as a cluster-formation algo-
rithm. Neural Computation, 2, 355-362.
58. Lang, K., Waibel, A. and Hinton, G. E. (1990) A Time-Delay Neural Network Architecture for Isolated
Word Recognition. Neural Networks, 3, 23-43.
59. Hinton, G. E. (1989) Connectionist learning procedures. Articial Intelligence, 40, 185-234.
5
60. Waibel, A. Hanazawa, T. Hinton, G. Shikano, K. and Lang, K. (1989) Phoneme Recognition Using
Time-Delay Neural Networks. IEEE Acoustics Speech and Signal Processing, 37, 328-339.
61. Hinton, G. E. (1989) Deterministic Boltzmann learning performs steepest descent in weight-space.
Neural Computation, 1, 143-150.
62. Touretzky, D. S. and Hinton, G. E. (1988) A distributed connectionist production system. Cognitive
Science, 12, 423-466.
63. Hinton, G. E. and Parsons, L. A. (1988) Scene-based and viewer-centered representations for comparing
shapes. Cognition, 30, 1{35.
64. Hinton, G. E. (1987) The horizontal-vertical delusion. Perception, 16.
65. Plaut, D. C. and Hinton, G. E. (1987) Learning sets of lters using back-propagation. Computer Speech
and Language, 2, 35{61.
66. Hinton, G. E. and Nowlan, S. J. (1987) How learning can guide evolution. Complex Systems, 1,
495{502.
67. Fahlman, S. E. and Hinton, G. E. (1987) Connectionist architectures for Articial Intelligence. IEEE
Computer, 20, 100{109.
68. Sejnowski, T. J., Kienker, P. K., and Hinton, G. E. (1986) Learning symmetry groups with hidden
units: Beyond the perceptron. Physica D, 22, 260{275.
69. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986) Learning representations by back-
propagating errors. Nature, 323, 533{536.
70. Kienker, P. K., Sejnowski, T. J., Hinton, G. E., and Schumacher, L. E. (1986) Separating gure from
ground with a parallel network. Perception, 15, 197{216.
71. Ackley, D. H., Hinton, G. E., and Sejnowski, T. J. (1985) A learning algorithm for Boltzmann machines.
Cognitive Science, 9, 147{169.
72. Hutchins, E. L. and Hinton, G. E. (1984) Why the islands move. Perception, 13, 629{632.
73. Hinton, G. E. (1984) Parallel computations for controlling an arm. The Journal of Motor Behavior,
16, 171{194.
74. Ballard, D. H., Hinton, G. E., and Sejnowski, T. J. (1983) Parallel visual computation. Nature, 306,
21{26.
75. Hinton, G. E. (1979) Some demonstrations of the eects of structural descriptions in mental imagery.
Cognitive Science, 3, 231-250.
76. Hinton, G. E. (1978) Respectively reconsidered. Pragmatics Microche, May issue.
Refereed Conference Papers
77. Srivastava, N., Salakhutdinov, R. R. and Hinton, G. E. (2013) Modeling Documents with a Deep
Boltzmann Machine. Uncertainty in Articial Intelligence (UAI 2013)
78. Graves, A., Mohamed, A. and Hinton, G. E. Speech Recognition with Deep Recurrent Neural Networks.
IEEE International Conference on Acoustic Speech and Signal Processing (ICASSP 2013), Vancouver.
79. Dahl, G. E., Sainath, T. N. and Hinton, G. E. (2013) Improving Deep Neural Networks for LVCSR
Using Rectied Linear Units and Dropout. IEEE International Conference on Acoustic Speech and
Signal Processing (ICASSP 2013), Vancouver