2013 CRA-W Graduate Cohort Workshop Finding a Research Topic Carla Brodley Professor and Chair, Department of Computer Science Tufts University (with credits.

2013 CRA-W Graduate Cohort Workshop

Finding a Research Topic

Carla BrodleyProfessor and Chair, Department of Computer Science

Tufts University

(with credits to Lori Pollack and Padma Raghavan)

Academic Academic HistoryHistory

• Started graduate school, UMASS…………….Fall 1988• Ph.D. awarded………………………………….Aug 1994• Started as Assistant Professor, Purdue….….Nov 1994• Promoted to Assoc. Prof. w/ tenure ………Spring 2000• Started as a Full Professor, Tufts ..………..…Fall 2004• Department Chair, Tufts……………………….Fall 2010

The Thesis EquationThe Thesis Equation

Topic + Advisor = Dissertationn

What is (CS) Research?What is (CS) Research?

• the systematic investigation into and study of materials, sources, etc., in order to establish facts and reach new conclusions

Oxford dictionary

– Experimental scientific research: • Observe a problem• Formulate a hypothesis• Develop a strategy to solve problem

based on hypothesis• Perform experiments and demonstrate

conclusive evidence• Interpret results

What is (CS) Research?What is (CS) Research?

• the systematic investigation into and study of materials, sources, etc., in order to establish facts and reach new conclusions

Oxford dictionary

Research is not knowing the answer or how to get it

– Theoretical scientific research:• Identify an open question• Formulate a hypothesis• Prove hypothesis

What is CS Research? What is CS Research? Example from Machine LearningExample from Machine Learning

ClassificationClassificationk-Nearest Neighbork-Nearest Neighbor

oo oooo

ooo

o

xx xxxxxxx

?

ClassificationClassificationk-Nearest Neighbork-Nearest Neighbor

oo oooo

ooo

o

xx xxxxxxx

?

ClassificationClassificationk-Nearest Neighbork-Nearest Neighbor

oo oooo

o

o

oo

xx xxxxxx

x?

Assign majority class of the k nearest neighbors

What is CS Research? What is CS Research? Example from Machine LearningExample from Machine Learning

• Observe a problem: Performance of k-NN is little better than random guessing for a particular dataset

• Hypothesis: Classification accuracy will improve if I can find and eliminate irrelevant and noisy features.

• Strategy: Develop a feature selection algorithm: eliminate features with low correlation with the class label

• Evaluation/Evidence: Implement and compare accuracy of original k-NN to new feature selection k-NN across a large number of data sets.

• Interpret results: Feature selection improves performance in M of the N datasets, …next steps?????

So, what isn’t PhD research?So, what isn’t PhD research?

How do I choose a topic How do I choose a topic area for my research?area for my research?

• Whose interest do you need to grab?– You– Your advisor– Your research community

• Love your topic!– Sets the course for your next 2-3 years– Determines, in part, opportunities offered

to you upon graduation– May work in same/related area for years

More Things to ConsiderMore Things to Consider• What are your strengths? weaknesses?

– Programming, design, data analysis, proofs– Key insights versus long/detailed

verification/simulation• What drives you? bores you?

– Technology, puzzles, applications, interdisciplinary• Do you (i.e., your advisor) have funding for you

to work in the area?– Working as a TA– Working as an RA– Having university/college, government, industry, etc…

fellowship/scholarship/grant

Which comes first?Which comes first?Advisor or Topic Area?Advisor or Topic Area?

• For many people “advisor before topic”– Meet faculty member with compelling research

interests• For some people “topic before advisor”

– Need a guide in an area already of great interest to you

• Want an advisor – Knowledgeable about your topic

• Interdisciplinary topics may require >1 advisor– With compatible working style (e.g., solo vs team)– With lots of research ideas– With strong interest in working with PhD students

Focusing from Area to Focusing from Area to TopicTopic

• Area = subfield– architecture, theory, AI, high performance

computing, or interdiscplinary– Is it important? Timely? Jobs in the area?

• Topic = specific open problems in subfield– Theory: provably better algorithm– AI: Improving a machine learning algorithm– Architecture: multicore cache design– HPC: parallel algorithm, scheduling scheme– Interdisciplinary: computer simulation of

tumor growth

Topic Scale and Topic Scale and ScopeScope• Scale

– Should have more than one open problem, or solving one should lead to another

– Should lead to more than one result/finding, some big, some smaller

• Scope – Too narrow, e.g., just analysis no

experiment, many not leave enough room

– Too broad, e.g., data mining, for what? why? too open ended

Selecting a Topic Selecting a Topic

• Moving from coursework to picking a topic is often a low point– Even for the most successful

students• Why?

– Going from what you know-coursework, to something new-research

– It is very important– There is no *one* ideal way, but

many good ways

7 Ways to Identify a 7 Ways to Identify a Good Research ProblemGood Research Problem

1) Flash of Brillance1) Flash of Brillance• You wake up one day with a new

insight/idea• New approach to solve an important

open problem

• Warnings:– This rarely happens if at all– Even if it does, you may not be

able to find an advisor who agrees

2) The Apprentice2) The Apprentice

• Your advisor has a list of topics• Suggests one (or more!) that you can

work on• Can save you a lot of time/anxiety

• Warnings:– Don’t work on something you find

boring, fruitless, badly-motivated,…– Several students may be working on

the same/related problem

3) The Extended Course Project3) The Extended Course Project

• You take a project course that gives you a new perspective

• The project/paper combines your research project with the course project– One (and ½) project does double duty

• Warnings:– This can distract from your research

if you can’t find a related project/paper

4) Redo … Reinvent4) Redo … Reinvent

• You work on some projects – Re-implement or re-do; Evaluate– Identify an improvement, algorithm,

proof

• You have now discovered a topic

• Warnings:– You may be without “a topic” for a long time– It may not be a topic worthy of a doctoral

thesis

5) Analyze Data5) Analyze Data

• You participate in more senior student’s evaluation study:– Help with data collection and analysis– Identify open challenges

• You have now discovered a topic• Warnings:

– You will have to agree on who works on identified open challenges

– It may not be a topic worthy of a doctoral thesis

6) The Stapler6) The Stapler

• You work on a number of small topics that turn into a series of conference papers

• You figure out somehow how to tie it all together, create a chapter from each paper, and put a BIG staple through it

• Warning:– May be hard/impossible to find the tie

7) The Synthesis Model 7) The Synthesis Model • You read papers from other subfields in

computer science or a related field• Look for places to apply insight from another

(sub)field to your own– E.g., machine learning to compiler optimizations

• Warnings:– You can read a lot of papers and not find a

connection– Or realize someone has done it already!– Or you have not made a significant impact in either

field

Tips and SuggestionsTips and Suggestions• Topic + advisor are both important• Keep a research ideas “journal” (wiki)• Keep an annotated bibliography (bibtex)• Follow your interests and passion

– Key driver for success and impact• Are you eager to get to work, continue working?

• If not really interested, adapt– Tedium or actual lack of interest and motivation?

When you’re stuck When you’re stuck at the startat the start• Read/present papers regularly to find

open research issues– Practice summarizing, synthesizing &

comparing sets of papers– Write your own slides for presentations

• Work with a senior PhD student on their research

• Try something….• Get feedback and ideas from others:

conferences, research internships, advisor’s idea

When you’re When you’re still still stuck…stuck…• Read a PhD thesis in your area

– Often contain an ‘open problems’ or ‘future work’ section

• Read your advisor’s grant proposals • Attend PhD oral exams and thesis defenses

– Understand how to formulate problems – Understand what constitutes a problem solution

• Assess your progress, with your advisor – Set goals per semester - Have you ruled out an

area? converged on an area? Chosen a topic for an exploratory research project?

When you’re stuck When you’re stuck againagain

• Divide your topic into milestones, and develop a plan to work on them one-by-one– Reward yourself when you finish a milestone – Publications and/or posters as milestones– Vary what you do during the day, but set aside blocks

of time for each activity

• Assess your progress regularly, with your advisor – Have you submitted a workshop paper? A term

project with documentation? A poster at a conference? A talk at a regional conf?

When you’re When you’re really really really really stuckstuck• Change research topics?

– May move you out of your advisor’s comfort zone of expertise

– Starting from “scratch” (e.g., need to learn the related work in a new area)

• Change research advisor?– May go through ‘shakedown’ period again– May or may not be better off

• But change can be invigorating– What’s hard? Need to recognize when things are not

working out and take action– Weigh consequences of changing and not changing

The Six Questions…. The Six Questions…. (from Paul Utgoff)(from Paul Utgoff)

• What research issue(s) interest you most? Why?• Who else has worked in this vein? What did they

accomplish? What can't they do?• What kind of progress would you like to see? Why?• Do you have an idea for making some such

progress? Explain.• What do you expect to discover from your

investigation?• How will your expected result(s) affect the research

community?

So how did I find my topic?So how did I find my topic?

• At ICML1990, I was irritated by – “Yet Another Learning Algorithm (YALA)”– Strategic selection of UCI benchmark datasets

to show YALA’s superiority• My idea: Given a dataset, select the “best”

algorithm automatically for that dataset• My next observation: Why should we assume all

parts of the data space have the same bias?• My next idea: Recursive automatic bias selection

Identify a research topic Identify a research topic and get started!and get started!

Great relevant article in ACM Crossroads, “How to Succeed in Graduate School: A Guide for Students and Advisors”, (part I, Dec 1994; part II, Feb 1995), available in ACM Digital Library