Vermelding onderdeel organisatie June 23, 2022 1 Literature Search http://www.pds.ewi.tudelft.nl/~iosup/Courses/ 2012_aiosup_lit_search.ppt IN 3305 Alexandru Iosup. Initial slides by Tomas Klos. Course manager: Peter van Nieuwenhuizen. Parallel and Distributed Systems Groep http://www.pds.ewi.tudelft.nl/
45
Embed
Vermelding onderdeel organisatie September 18, 2015 1 Literature Search iosup/Courses/2012_aiosup_lit_search.ppt IN 3305.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Vermelding onderdeel organisatie
April 21, 2023
1
Literature Searchhttp://www.pds.ewi.tudelft.nl/~iosup/Courses/2012_aiosup_lit_search.ppt
IN 3305
Alexandru Iosup. Initial slides by Tomas Klos. Course manager: Peter van Nieuwenhuizen.
Parallel and Distributed Systems Groephttp://www.pds.ewi.tudelft.nl/
Literature Surveys: At the Core of InnovationGiven a problem (topic of interest)Answer questions about it
• What solutions exist?• What is the most influential solution?• What is the rate of innovation in the field?
By surveying (understanding, interpreting, and summarizing) the body of related (scientific) knowledge.• Where and how can I innovate?
IN3305’s study goal “kennismaken met wetenschappelijke literatuur”
Innovation is a Vital Competitive Tool
• Innovation = novel application of knowledge• Innovation favors small (but efficient) countries• High-tech companies tend to be more innovation-intensive
Source: Economist Intelligence Unit, A new ranking of the world’s most innovative countries, April 2009, http://graphics.eiu.com/PDF/Cisco_Innovation_Complete.pdf
What is Novel?The Overwhelming Growth of Knowledge“When 12 men founded the
Royal Society in 1660, it was possible for an educated person to encompass all of scientific knowledge. […] In the last 50 years, such has been the pace of scientific advance that even the best scientists cannot keep up with discoveries at frontiers outside their own field.” Tony Blair, PM Speech, May 2002
19972001
19931997
Number of Publicatio
ns
Data: King,The scientific impact of nations,Nature’04.
The “Size” of a Research Topic
• Grid Computing• Billions of $ in research investment• 2,500 PhDs (my est.)• Over 15,000 scientific publications (my est.) in 15
Literature = output• Publish to conferences and journals• Peer-review (for conferences, journals):• (double) blind review:
Accept, with/without (major) revisionsReject
• Acceptance rate ratio, e.g., 25% (not bad)• (Nature: 10% articles are reviewed)• Time to print: up to 1.5 years for journals,
3-6 months for conferences• Measuring scientific output: “scientometrics”
Q What do you think about this situation?
April 21, 2023 12
Quality?
• Reputation: ACM, IEEE, Springer, Elsevier, MIT/Princeton/Oxford/… University Press
• SCIgen - An Automatic CS Paper Generatorhttp://pdos.csail.mit.edu/scigen/accepted (non-reviewed) for: 2005 World Multi-Conference on Systemics, Cybernetics and Informatics (another one: an Elsevier journal!)
April 21, 2023 13
Scientometrics• Scientometrics, “measuring and analyzing science”,• Bibliometrics, “study or measurement of texts and
information”• Citation analysis• Which papers cite a paper / does a paper cite?• Authority of countries, research groups, individual
authors, journals/conferences, individual paper
Q What is a citation?
• “Publish or perish”: quality vs quantity• (“80% of all published papers are not cited”)
Q Conference or journal? Which conference or journal?
April 21, 2023 14
Citation Databases
• Commercial• ScienceCitation Index (Web of Science/Inf. Sci.
Inst.)• Scopus (Elsevier)
• Free• Google Scholar: better coverage than ISI• CiteSeer (computer science)• ArNetMiner (computer science)• RePec (economics)
Data: King, The scientific impact of nations, Nature’04.
Citation rate per paper, norm.
Citation intensity=
#Citations/GDP
Comparing Groups or Individuals [1/3]• An idea: Google PageRank principle
• Web: network of sites, linking to each other• Science: network of papers, citing each
other
Time
World Wide Web’s Links Network
Academic Citations Network
Q What do you think about this approach?
April 21, 2023 17
Comparing Groups or Individuals [2/3]• Journals: Journal Impact Factor• Personal: h-index (Hirsch, 2005):
A scientist has index h if h of his/her N papers have at least h citations each, and the other (N − h) papers have no more than h citations each.g-index (Egghe, 2006): highest number g s.t. the first g most cited articles have attracted at least g2 citations.
• Extensions: e-index; group evaluation
Q What about conferences?Q Really, what is a citation?Q (unethical) How to abuse citation indices?
April 21, 2023 18
Journal Impact Factor (JIF)
• Many journals have no impact factor• JIF is the average number of citations in a given
year, to papers in a journal in the 2 previous years.
• For journal x, 2010
number of citations in 2010 to papers in journal xfrom the period 2008 – 2009
JIF (x, 2008) =Total number of papers in journal x
in the period 2008 – 2009
• What does an average value mean?
April 21, 2023 19
Journal Impact factors, 20042004 Science Journals Impact Factors (Bron: ISI)
0.001
0.01
0.1
1
10
100
0 1000 2000 3000 4000 5000
≥1 citation/publication(last 2 years)
JIF
Journal Rank
Highest JIF ~30
Very high JIF ≥15
April 21, 2023 20
CS impact factors, 20052005 Impact Factor CS Journals (Bron: ISI)
0.01
0.1
1
10
0 100 200 300
Journal Rank
JIF
Highest JIF ~8
Very high JIF ≥2
Highest JIF ~30
Very high JIF ≥15
CS All
Q What do you think about this situation?
Comparing Groups or Individuals [3/3]For Computer Science• Conference proceedings are to be preferred to
journals• ISI Web of Science and Elsevier Scopus are not good
impact indicators—poor, albeit improving, coverage• Google Scholar is a better impact indicator than ISI
WoS and Elsevier Scopus; ArNetMiner is reasonable• DBLP is a good, selective source, but has no citation
links• Expert knowledge is required to select the best
topical conferences and journals (regardless of their acceptance ratios and impact factors)Q Problems with this
approach?
April 21, 2023 22
Outline
1. From the IN3305 study goals:1. “kennismaken met wetenschappelijke
literatuur”2. To read or not to read?3. What is “scientific literature”? (input and
output)4. Measuring and assessing Quality5. Useful sites and tools6. On gaming the citation indices (unethical)7. Conclusion
April 21, 2023 23
Method To Find Sources
• Browse:• Google Scholar: http://scholar.google.com/• DBLP: http://dblp.uni-trier.de/• Others: TU Delft library tools
• Study author using Publish or Perish• Look at author homepages• Follow links and citations (forward and
backward)
April 21, 2023 24
Google Scholar
• “cited by”• Relevant authors• TU Delft SFX linking• Import into bibtex
April 21, 2023 25
Google Scholar at Work
April 21, 2023 26
April 21, 2023 27
Google Scholar at Work
From home: use vpn!
April 21, 2023 28
April 21, 2023 29
DBLP
• “lists more than one million articles” (april 2008)• Indexes:• Authors• Now also “Faceted search”,