Professors Sergio Verdú (Princeton; currently on leave at Stanford) and Tsachy Weissman (Stanford) have started an information theory forum at Stanford, which includes a weekly meeting for students and faculty to present tutorials on topics related to the Science of Information and a blog on information theory (the Princeton-Stanford b-log): http://blogs.princeton.edu/blogit/
Blog: Student/Postdoc Spotlight
Thursday, March 07, 2013
We sat down with Sheila Rosenberg during a break in the student-postdoc research workshop that took place in July 2012. This is what she had to say about her interactions with the Center.
My name is Shelia Rosenburg and I’m a Postdoc in Todd Coleman’s lab at UC San Diego. I love the lab, I mean, one thing that I think is amazing is that Todd’s lab is kind of like a mini version of the Center. What I like about the lab and the Center is the unique way different people approach the same question or tackle a problem. I’m enjoying being around engineers so much because they are more theoretical in general. They are really good at abstraction and they frame questions differently. I think even coming at things with a different language can be very valuable because then you get into a discussion, and you ask the question - "Well what do you really mean by that?" and I think it forces you to go deeper into the issues because you actually have to make sure you understand each other, and you have to make sure you’re on the same page to move forward.
Here in the Center the potential for collaboration seems really great because it is the same kind of mentality that people bring their different skill sets. I again am impressed at how open minded people are here among the students in the Center. I’m really impressed people are so willing to share and have an exchange. I think we were talking about one of the fundamental things is having the chance to learn new skills. People here seem very willing to share that and I think that’s going to enhance any kind of collaborative project.
If you’re not only working alongside someone, but you both get to learn about the other person’s area I think that is mutually beneficial. The science will be better and I think we'll be able to ask bigger and more important questions because we have a unique team to tackle the problem.
So many of the themes and the conceptual ideas being discussed from a more information theory perspective are very applicable to neuroscience. Because again we’ve been talking about this a little bit; the way the brain works, the nervous system is all about taking in information figuring out how to sort it figuring out how to analyze it, figuring out how to store it.
I really like things like this [research workshop] because i feel like learning the language and some of the concepts that already exist in this area of studying information. Just by having an introduction to these topics for neuroscientists is very helpful.
And, again having the chance to sit down with people who are in theory and say "here’s what we'd like to know about the brain". Because I feel like the theoreticians I’m meeting are really good at saying "Well what you are talking about...that sounds like this type of problem or that type of problem" And, importantly, they are not trying to box my idea into a certain similar IT problem.
Being able to find these commonalities is crucial, and then for me, there are definitely answers for me that require a quantitative and computational approach. Not that I don’t have the skills for it, I don’t even know where to begin necessarily. Finding people who say "oh that sounds a little like this strategy would be useful" I think that there is so much potential here to do this; to really go after these big questions of learning and memory and perception and then also direct that toward eventual therapeutic applications.
For sure, there is inherent risk in going after big new things, and I think for various reasons like funding and other issues people are hesitant to take this risk. However, if you have the backing of a center, and you have a group of people willing to go in on it together, then your potential to make headway on something challenging can be higher probability of success.
Electrical and Computer Engineering, Carnegie Mellon Univesity
Thursday, November 29, 2012
We sat down with Pulkit Grover during a break in the student-postdoc research workshop that took place in July 2012. Pulkit was a post-doctoral scholar at Stanford during the time of this interview. He is now an Assistant Professor in Electrical and Computer Engineering at Carnegie Mellon University. This is what he had to say about his interactions with the Center.
Pulkit, tell us about your impressions of the Center and what interactions you have been involved in terms of research and education
I think the effort of the Center developing a list of grand challenges is great but towards addressing those grand challenges and even in the process of coming up with those grand challenges, the discussions that are happening across different communities is just wonderful.
This very discussion is happening during our research workshop going on now; we were just talking about issues in neuroscience and how measurements made in neuroscience today are being influenced or can be influenced by a better understanding of what those measurements are being used for. This all boils down to goal oriented communication in a way, which is very interesting as an area of application for idea.
My background is in information theory. Personally for me, developing during a graduate student and now a post-doc, this development from taking an information theory class and information theory ideas to understanding where all information theory can go to is a very satisfying thing.
For me there were two interdisciplinary interactions that happened during the workshop today and last night.I was talking to a neuroscientist from UCSD she is a post doc from Todd Coleman’s group. She was telling me about how measurements are made [in the brain], and then on the other hand there was quite a bit she had to understand the language I was trying to speak and i had to understand a lot of words she was using. In an informal environment it was very easy to do, you could never do it during a talk. I could probe her on questions, "so I’m pretty stupid in neuroscience, I’ve watched movies and read science fiction. But not really any books. So tell me what this means" And she was very ok with explaining it to me in a child’s language.
I also had a promising interaction with the computer scientist from MIT who works in cryptography and we were trying to develop ideas on how to look at boolean operations and security in systems. There is an understanding of security within information theory, which has for good and not so good reasons not had much application in practice, while computer science cryptography has been extremely successful in getting it into practice. Now what is the reason?
In order to understand this, it is important for me to understand how computer scientists perceive security. And then try to see how information theoretic security can apply to that area.
We actually wrote out a tiny problem formulation, which we couldn’t solve in one night!
But, the hope is that we can do it in the next year or at least begin to address that with let’s say a seed grant.
Do you feel it is important for graduate student to interact on an interdisciplinary level?
Well, faculty get to interact with each other at a level where they interact across disciplines all of the time, if they want to. Graduate students don’t even have that access because they don’t have those links established yet. Sometimes credibility is an issue. So this environment [The Center has built] is really nice in being able to get graduate students and post docs together from different streams.
I love the center, its genuinely amazing. It’s often said that interdisciplinary collaboration and interactions are very useful but what’s happened over the years is that fields have matured pretty much independently And it’s not over the process of the last ten years it’s over the past thirty or forty years.But the emphasis that we are now laying on big data, and bringing on new insights and new ideas or new problems that challenge us in different communities, information theory community, computer science community, and life science community, that challenge might push us again to come together in a way that different communities came together during world war two and afterword’s.
I was reading this book called 'The Information', it’s a beautiful book! It’s a genuinely beautiful book written by James Gleik, which describes the history of information theory and interactions between Turing and Shannon at the time. And it was just excellent and that was again interdisciplinary interactions. Shannon worked on building guided missiles during world war two, and Turing worked on breaking codes. And these are interdisciplinary activities, once you build something it has to be interdisciplinary. It helps you enhance your perspectives on how to abstract, or it helps you model these sort of things, once you build these things. And I think that interaction of Turing with practice in general has weakened over time. Something like the Center can certainly help build up that connection again.
Tuesday, April 17, 2012
Who inspires you most and why (could also be what event or person first captured your interest to study CS)?
I knew I enjoyed working with computers before going into CS, but only as a hobby rather than an academic discipline. Like many others, I held the common misconception that CS and programming are equivalent. This misconception was obviously blown away once I took a few CS classes and met other undergrads who were engaged in research and realized the awesome cross-disciplinary applications of CS.
What are you interested in studying/researching in graduate school?
I'm interested in machine learning and natural language processing.
What are your current long-term career goals?
It's hard to say at this point, although at the moment, I would like to continue in academia. What I know for sure is that I would like to be in a position where I can engage in research and also mentor students who think they may be interested in pursuing CS to any degree. Personally, my mentor has played a huge role in helping me discover the parts of CS and research that I find most exciting and encouraging me to pursue those passions in graduate study, and I would love to have the opportunity to influence other students the way he has influenced me.
What suggestions do you have for others on applying to scholarships/fellowships such as the NSF Graduate Research Fellowship?
The most obvious piece of advice that I can give is to *apply*! You have nothing to lose and everything to gain. Start early and don't be embarrassed to ask anybody -- friends, family, fellow students, and especially professors -- for feedback on your ideas or essays. The NSF GRFP in particular makes it very clear what they're looking for in the application, so make sure you follow their guidelines to a tee. And for those underclassfolk who might be considering applying for these in the farther future: seek out research opportunities and establish close relationships with professors (these two usually go hand in hand). These things can only help.
Wednesday, March 14, 2012
Give us your background as a person and a student and how you eventually came to work on your Ph.D….what sticks out in your mind as big influences or important events that got you to this point?
I was born and raised in China, where I got my bachelor’s and master’s degrees. The main reason why I came to work on my Ph.D. is that I think it is necessary for me to get the highest degree when I am still young such that I will not be regretful after ten or twenty years. And I believe it will take me much more efforts to get this degree when I was over 40 or 50 than I do now. Also, I think my family has a big influence on getting me to the current point. My father is a professor in math. Because of this, I am also very interested in math related subjects. Doing research in Computer Science, especially machine learning, has a lot to do with math, and is tightly linked with people’s lives. So, I became very fascinated in this area and made the determination in getting a Ph.D.’s degree in this area.
Describe the field you work in.
I mainly work on Data Mining, Information Retrieval and Machine Learning. As the names suggest, data mining is mainly for digging knowledge out of the data, and information retrieval is more related to search engines, web intelligence, etc. Machine Learning can be considered as a general type of methods that can be applied to both data mining and information retrieval. I like these areas, because they are playing very important roles in our lives, and changing our life styles to a large extent. For example, I believe 20 years ago, nobody has ever considered to use Google to find the desired information. But now, it becomes extremely successful in facilitating our lives.
Tell us about the recent fellowships you were awarded and a finalist for (IBM / Facebook - and talk about your past internship with IBM and what you foresee in your upcoming efforts working with topics that are of interest to IBM and others)
Recently I was awarded the IBM Ph.D. fellowship as well as the Facebook fellowship finalist. I did my internship at IBM in 2010 summer. At that time, they were working on a project related to Twitter. Based on this project, I proposed method for Twitter categorization and filed a patent. They like my works over there. So, I think this is one of the reasons why they give me the fellowship this year. But I cannot foresee which topics they may be interested in now. It highly depends on what project that group gets.
and then how does your research further the knowledge of those fields, and the potential applications for society, business, etc?
I am very interested in applying machine learning methods to solve some real world problems, such as webpage clustering, market targeting, document retrieval. For example, in one of my recent works, I improved the hashing algorithm for document retrieval. Also, in IBM, I designed an algorithm to automatically find potential company partners. I believe these topics are very useful for society and business.
Tell us about your experience as a graduate student – what have you enjoyed, what have you found challenging, and what advice would you give to undergraduates considering graduate school, especially those interested in going into science or engineering fields.
Understanding the outside world is an eternal topic for human beings. I enjoyed the feeling that I am pushing this understanding forward, and contributing to make the computers and webs smarter. There are too many challenges in doing research, especially when you came across a novel application. You may totally have no idea what you can do in that direction. These challenges will knock you down, but may also arouse your interests. When coming across challenges, talk to your advisors, your collaborators, and try to find the answers in previous papers. This process could be tough. But giving up easily is never a good option.
For undergraduates, I would suggest you to ask yourselves if you truly like doing research or not before deciding to go to a graduate school. If you like it, then, you can consider to pursue a Ph.D.’s degree. If you are not sure about that, you can first go to a graduate school and work on some specific research areas for a couple of years to see if you become interested in that or not and decide whether you should continue your studies for the Ph.D.’s degree. When choosing graduate schools and advisors, several factors, such as good discussion environments, supportive advisors, reputations of the program, are all very important for making the decision.
Monday, February 13, 2012
Project: Predicting Marked Code-switching in African Languages
Give us a little bit of your background as a person and a student and how you eventually came to work on your undergraduate in this field…
I am a junior Computer Science major from Ondo State, Nigeria, studying at Howard University. I came about working on my research project as a result of my Introduction to Algorithms class which I took with my current research advisor, Dr. Rwebangira. Because of my performance in his class, Dr. Rwebangira approached me about his research work. This project involved aspects of Machine Learning and Natural language Processing - subjects I find very interesting. Therefore, I grabbed this golden opportunity to learn about these fields.
What sticks out in your mind as big influences or important events that got you to this point?
Upon graduation from senior secondary school (the Nigerian equivalent of high school), I had no idea of what I wanted to study in college. After several days of contemplating on different career choices, I finally settled on Computer Science. It was lucrative and it required a fair level of creativity. I considered it a fairly good fit so I took a chance- a huge gamble looking back now.
My first programming class, I was very nervous because I had never programmed before so I was scared of failing. However, over the course of the first two weeks, I eased into programming and I have not looked back since. In fact, my passion for programming and Computer Science in general has only grown. Hence, a major turning point in my life was a very lucky gamble.
How would you describe the field you work in, and how does your research further the knowledge of that field, and the potential applications for society?
My research work involves several aspects of Machine Learning and Natural Language Processing.
I believe it is potentially beneficial to the society because it attempts to train the computer to understand the certain aspects of everyday social interactions. In order for machines to better serve their users (humans), I believe it is only natural that they understand them.
As you know our early efforts in the Center are to bolster the emergence of a new science of information that incorporates the knowledge of a number of related areas.
How do you feel that your project links in with the Science of Information….and are there areas of Science of Information you feel could benefit your own work?
My research project has a direct link with the Science of Information as it essentially involves extracting meaningful information from data.
Also, since the Science of Information is such a broad subject that encompasses aspects of different fields including Machine Learning, I believe knowledge of different aspects of the Science of Information will help me devise and explore several different approaches in my research work.
Tell us about your experience as an undergraduate student – what have you enjoyed, what have you found challenging, and what advice would you give to high school students considering college, especially those interested in going into science.
I have had a great experience as an undergraduate. I have become a good software developer and have also developed a great network of hopefully lifelong friends.
As regards challenges, I initially had problems balancing my academic workload with extracurricular involvements. However, over time, I have learned to clearly set my priorities and to only take on as much as work as I can handle.
Also, to high school students considering studying science, I believe science is at the forefront of change today. All the “cool things” around today- Siri, Google, the iPad, the iPhone- is as a result of science. Therefore, by going into science, one will be a driver of change- the creator of “cool things.”
What would you like to do next? In the future?
I would ideally like to work during the summer of 2012 with faculty at one of the Center partner's, and then finish my research work at Howard next winter.
In the long-term, I plan to pursue a PhD in Computer Science to further explore the field of Machine Learning and Natural Language Processing. Consequently, I plan to start my own company, hopefully based on my research work.
Tuesday, November 08, 2011
I joined Claudio Aguilar’s lab here at Purdue University. He has two different systems that he works on simultaneously: cell biology with yeast and cell biology with mammals. So basically, yeast is a very good system to work with as almost all the proteins in yeast are conserved in the mammalian cells. For example, different discoveries that have been made with yeast are applicable to mammalian cells as well. It is so easy to work with, easy and fast to get all the data, so I like to work with the yeast system.
My research is focusing on a protein (Epsin) that is important for endocytosis, and we see how that protein is absolutely required for the cells to survive. So the interesting thing is, the endocytic function of the protein is not required for its survival. So there’s something else going on. The story starts when Claudio figured out exactly what’s going on – why the cell’s need that particular protein to survive.
I followed up with Dr. Aguilar’s work and the interesting thing I found is that there is a region in the protein that is made up similar kinds of amino acids – proteins are made up of amino acids - and this region of the protein has the same consecutive amino acids for a stretch. The reason this is important is because in several neuro-degenerative disorders like Huntington’s Disease, this stretch of amino acids expands and makes the protein pathogen. So it’s not the same protein. Huntington’s Disease has a different protein that undergoes its own expansion. This amino acid is called glutamine and the process is polyglutamine expansion. What it seems is that this poly-glutamine region is important for certain functions of the protein to help it maintain viability, proper cell division and so on.
The reason this is very important is because when people try to understand polyglutamine mediated neuro-degenerative disorders they usually look at the expanded protein, but nobody has ever looked at why the small stretches are there in the first place. That is what we are trying to do. We are using this protein to understand the physiological relevance of the small polyglutamine stretch that is there in the first place and then probably move on from there and see, when it undergoes expansion, what changes: if there’s any gain of function, loss of function and so on.
Yeast cells are a diverse population, so even when something is wrong with them, you need to look at thousands of cells to be sure that it is really going on, because even in normal cells you will find one cell out of, say, 5000, in which something is wrong – but that is not really relevant. So in order to do this, we need to look at these thousands of cells and then look at their perimeter outline, and conduct image analysis. We have to do this manually and it takes a lot of time. We feel that if there were an automated approach to the image analysis, then we could do it much faster.
Information Theory methods and tools could really help us we think. And, such as analysis would be not only faster, but since we’re currently doing the image analysis manually, there might be some information that we’re missing. An automated system of image analysis would not miss these critical pieces of information. So bridging information theory with cell biology and the resulting image analysis would be become more information-rich and we would potentially get a lot more relevant information from an automated system.
I also realize there is one more thing in which image analysis automation could help us with. Epsin is made up of a large number of amino acids. We know only about 10% of what a whole sequence does, of what the individual amino acids do. Perhaps by looking at different sequences of different proteins we might be able to understand more about what each amino acid or what each group of amino acid is responsible for and so on. Basically what this shows is that if you have a lot of epsin, it promotes cells to become cancers. We feel that bridging information theory with cell biology is necessary in order for the science to progress at this point."
Tuesday, September 27, 2011
"Bridging Electrical Engineering and Neuroscience"
Here in my lab at the University of Illinois, in cooperation with Todd Coleman and Rhanor Gilette, who are my advisors, we look into bridging neuroscience and electrical engineering. What we think is that in the brain one of the theories of addiction assumes that there is a rewiring in the brain; or that addiction results in the same pathway as learning, and the same pathway as reward for feeding habits. This is the hypothesis we are using with our animals (Sea Slugs). We are trying to see, when we have these addicted animals, if what is happening in the reward network and in the reward pathway is really changing. We have these very simple animals (sea slugs) that have very simple brains and they have a set of behaviors; they are not as complex as humans that have many behaviors. They have a very finite set of behaviors and we try to understand how their simple brains coordinate that behavior. The slugs are a good choice for us to study compared with the complex human brain because those animals have a limited number of neurons, they are bigger than human neurons, and you can really track down what's activating what and what's directing what behavior.
We have this idea that an animal is trying to solve or optimize his cost function, we are using both Markov processes and optimization theories - so that we can model the animal's behavior. What we want to do is to observe both the normal animals and then have an addicted animal and see how that cost function changes. So if the animal is going around - and we work with sea slugs, they are natural predators, so if we can alter its behavior through addiction we want to understand how that cost function is changing and how the neural processes are changing. So that's when we are bridging electrical engineering with the math of Markov processes and inverse optimal control theory with neuroscience and neurobiology.
I think one of the first things that would come out is if we could show how these optimizations are happening and how the cost function and the rewards of the animal change when he is addicted versus when is not addicted. These algorithms we are working on in collaboration with other students, I think that we really could bring a different approach to the way we analyze. We are also planning on bringing neural recordings so that it will also be unique.
The model we are using - the inverse optimal control model - can be used not only for this specific animal, but it is a very generic, per se, algorithm that can be applied to different animals and different models.
I've found that when you start working with inter-disciplinary research, you have to be comfortable with going beyond your boundaries. For me, as an electrical engineer, I think everything in terms of logic and math - either yes or no, on or off, binary in essence- and when you go into science, you have this continuum that you have to be comfortable working with and experiments that are not a simulation in a computer and you have to be there present; it can work or it can fail and you just have to be comfortable with that. But it is also very challenging; it is one of the things I enjoy the most: just being able to, for me as an electrical engineer, doing dissections and separating neurons. I think it's really amazing. I would welcome discussion, questions, and comments on my work in the Center. Thanks!