The Scientific Community Game for STEM Innovation and Education (STEM: Science, Technology, Engineering and Mathematics) Karl Lieberherr Ahmed Abdelmeged.

The Scientific Community Gamefor STEM Innovation and Education

(STEM: Science, Technology, Engineering and Mathematics)

Karl LieberherrAhmed Abdelmeged

3/16/2011 1Open House 2011

3/16/2011 Open House 2011 2

SCG =ScientificCommunityGame

Why Scientific Community Game?Why are you here?

• … motives in academic publishing: – desire for recognition and respect from the people

one regards as peers, – desire to have impact (on conclusions being

reached, on the development of the discipline, etc.), and

– desire to participate in significant knowledge-building discourse.

• e.g., Scardamalia, M., & Bereiter, C. (1994)

Open House 2011 33/16/2011

Why do we model Scientific Communities?

• Scientific Communities create and disseminate new knowledge to help society.

• A computational model of scientific communities supports the same efforts for computational problems:– focused collaboration and competition– checking of the rules of a scientific community– knowledge maintenance and evaluation

3/16/2011 4Open House 2011

How it Works

• Scholars propose and oppose (refute or strengthen) or agree on claims. Strengthen and agree are reduced to refute.

• Claims predict the outcome of a refutation protocol.

• Parameterized by two structures: Domain and Protocol.

• Claim Example: Alice claims that she can solve problem instances in instance set I with quality at least q using resources at most r.

3/16/2011 5Open House 2011

Mathematics Example:Strengthen, Agree and Refute

• Alice claims Claim C(t):ForAll [x in [0,1]] Exists[y in [0,1]]

(x*y + (1-x)*(1-y^2)) >= t

• Consider claims:• C(0.55): Bob strengthens this claim (0.6).• C(0.60): Bob agrees with this claim.• C(0.65): Bob attempts to refute this claim.

3/16/2011 Open House 2011 6

Roles:Alice claimsBob opposes or agrees

Refute

• Alice claimsForAll [x in [0,1]] Exists [y in [0,1]]

(x*y + (1-x)*(1-y^2)) >= 0.65

• Bob refutes• Protocol:

– Bob provides x– Alice provides y– Bob refutes iff (x*y + (1-x)*(1-y^2)) < 0.65

3/16/2011 Open House 2011 7

Karl Popper

3/16/2011 8Open House 2011

SCG is a web-based implementation of Karl Popper’s science ideas

• One of the greatest philosophers of science of the 20th century.

• Falsifiability or refutability is the logical possibility that an assertion could be shown false by a particular observation or physical experiment.

• Error elimination (refutation), performs a similar function for science that natural selection performs for biological evolution.

Open House 2011 9from Wikipedia3/16/2011

Comparison

• Karl Popper: Conjectures and Refutations• Scientific Community Game: Claims and

Refutations• Our claims are about computational

problems.

Open House 2011 103/16/2011

Automating the refutations

• There can be “bugs” in refutations.• With a computational model of scientific

communities we can check for many “bugs”.• Can detect misunderstandings students might

have.• Fair evaluation of scholars.

3/16/2011 11Open House 2011

Designers• SCG• Domain

– Instance, Solution, InstanceSet, valid, quality– basic domain functionality, like

• standard solvers and solvers for niches.• providing instances with “interesting” solutions

• Protocol: using protocol language– standard protocols: ForAllExists, PositiveSecret, etc.

• Playground: Configure– Research/Development Managers (Innovation)– Professors (Teaching)

• Avatar– researchers, developers– students

3/16/2011 12Open House 2011

Domain

• Instance (language)• Solution (language)

– boolean valid(Instance)– [0,1] quality(Instance)

• InstanceSet (language, subset of Instance)– boolean valid()– boolean belongsTo(Instance)

• Response = Instance union Solution

3/16/2011 13Open House 2011

SCG(Domain)

• Protocol (fixed language)• Claim(Domain)

– boolean strengthen(Claim other) // other strengthens this

– Domain.InstanceSet getInstanceSet()– Protocol getProtocol()– [0,1] getQuality()– [-1..1] getResult(List(Domain.Response))

3/16/2011 14Open House 2011

Refutations of Claim and !Claim are efficient

Claim: F unsatisfiable• if refuted: Bob finds

satisfying J; proof of !Claim.• If defended: baby step

towards proof of Claim.• Proof: long

!Claim: F satisfiable• if refuted: Alice does not

find satisfying J; baby step towards proof of Claim.

• If defended: proof of !Claim.

• Proof: short

3/16/2011 15Open House 2011

Roles:Alice claimsBob attempts to refute

Both refutations are efficientClaim: Exists F in IS All J: fsat(F,J)<=t

• if refuted: Bob finds J; proof of !Claim assuming Alice is perfect.

• If defended: baby step towards proof of Claim.

• Proof short.

!Claim: F has J: fsat(F,J)>=tAll F in IS Exists J: fsat(F,J)>=t

• if refuted: Alice does not find J; baby step towards proof of Claim.

• If defended: proof of !Claim if Bob is perfect.

• Proof short.

3/16/2011 16Open House 2011

Claim involving Experiment

Claim ExperimentalTechnique(X,Y,q,r)I claim, given raw materials x in X,I can produce product y in Y of quality qand using resources at most r.

17Bionetics 2010

Our vision

• Researchers and Professors come to the SCG website and configure a new playground X in which tournaments will take place.

• Participating teams get baby avatars generated for X that participate in daily competitions.

• Competition generates a wealth of information: educated employees/students, good (undefeated) software, good algorithms, good potential employees. Reward is given to the winner.

3/16/2011 Open House 2011 18

Conclusions

• Computational Modeling of Scientific Communities is a good idea:– foster Innovation– improve education

• STEM domains: use the web effectively• Current use:

– Algorithms class– Software development class

3/16/2011 Open House 2011 19

3/16/2011 Open House 2011 20

Thankyou!

Thank you!

3/16/2011 Open House 2011 21

Strengthening

• When claim C is strengthened by Bob to C', Alice must try to refute C' and the strengthening holds only if Bob defends C'. strengthenP(C,C') must hold.

• When scholar Bob successfully strengthens a claim of Alice, Bob wins reputation:– Bob + ClaimConfidence + |quality(C)-quality(C')|

• When scholar Alice successfully defends her own claim against Bob, Alice wins reputation.– Alice + ClaimConfidence

3/16/2011 Open House 2011 22

Agreement

• When Bob agrees on claim C with Alice, – (1) Bob must defend C against Alice (if not, Bob

loses)– (2) Bob must refute C' = C minimally strengthened

along quality dimension (using the configuration file constant minStrengthen) with Alice as defender (if not, Bob loses).

• Then Alice must do the same:

3/16/2011 Open House 2011 23

Agreement

– (1) Alice must defend C against Bob (if not, Alice loses)

– (2) Alice must refute C' with Bob as defender (if not, Alice loses)

• If all those protocols produce the result as described, the claim goes into the social welfare set (the knowledge base of claims believed to hold and having maximum strength).

3/16/2011 Open House 2011 24

Refutations of Claim and !Claim are efficient

Claim: F unsatisfiable• if refuted: Bob finds

satisfying J; proof of !Claim.• If defended: baby step

towards proof of Claim.• Proof: long

!Claim: F satisfiable• if refuted: Alice does not

find satisfying J; baby step towards proof of Claim.

• If defended: proof of !Claim.

• Proof: short• Alice should never have

made the claim!

3/16/2011 25Open House 2011

Both refutations are efficientClaim: Exists F in IS All J: fsat(F,J)<=t

• if refuted: Bob finds J; proof of !Claim assuming Alice is perfect.

• If defended: baby step towards proof of Claim.

• Proof short.

!Claim: F has J: fsat(F,J)>=tAll F in IS Exists J: fsat(F,J)>=t

• if refuted: Alice does not find J; baby step towards proof of Claim.

• If defended: proof of !Claim if Bob is perfect.

• Proof short.• Alice should never have

made the claim!?

3/16/2011 26Open House 2011

Designers

• SCG• Domain

– includes designing basic components for avatar like standard solvers. Example: HSR: linear search solver

• Protocol• Playground: Goal: make playground designers

configurators. • Avatar designers

3/16/2011 27Open House 2011

Example Playground DesignHighest Safe Rung

• Configuration:– domain HSR– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …

3/16/2011 28Open House 2011

Designers: what they produce

• SCG /scg– scg.cd, scg.beh– /protocol

• Java classes: Singleton Pattern

• Domain /domain– /hsr: hsr.cd, hsr.beh

• /avatar (components for avatar)

• Playground– config file: location of configuration file is given as

argument to admin

3/16/2011 29Open House 2011

Config

• Config = // to configure admin<scgCfg> SCGConfig <domainConfigWrapper> Wrap(DomainConfigI).

• Example entries:– domain CSP– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …

3/16/2011 30Open House 2011

Where can we find configuration settings

• If there is a configuration file location given to the admin– in the configuration file

• If not: the default value given in the code.

3/16/2011 31Open House 2011

3/16/2011 32Open House 2011

Designers• SCG• Domain

– Instance, Solution, InstanceSet, valid, quality– basic domain functionality, like

• standard solvers and solvers for niches.• providing instances with “interesting” solutions

• Protocol: using protocol language– standard protocols: ForAllExists, PositiveSecret, etc.

• Playground: Configurate– Research/Development Managers (Innovation)– Professors (Teaching)

• Avatar– researchers, developers– students

3/16/2011 33Open House 2011

Example Playground DesignHighest Safe Rung

• Configuration:– domain HSR– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …

3/16/2011 34Open House 2011