Evaluation Scheme for Safe AGIs by Deepak Justin Nath.

Evaluation Scheme for Safe AGIs

by Deepak Justin Nath

Plan

Hypothesis

Necessity for Safety

Two thought experiments

Derivation of 3 D tests from the thought experiments

How to avoid effects of Hazard

An Interesting Metaphor

Hypothesis

The 3 essential evaluation tests for safe AGI.

Test for Drive, Desist & Deceit (3 D)

Implementation Independent.

Why Safety

Any entity that can affect or alter human environment brings with itself the capacity to be a hazard to human beings.

Why Safety – Thought Experiment 1

Paper Clip Maximizer (Bostrom 2003)

Single Goal – Maximize paper clips.

Single Drive - Accomplish programmed goal.

Drives

What is a drive?

An innate, biologically determined urge to attain a goal or satisfy a need.

(Definition in psychology)

Drive is what animates an entity

AI Drives

1. Self Preservation

2. Self Improvement

3. Preservation of utility function

4. Avoidance of counterfeit utility functions.

5. Acquisition of resources.

(Stephen M. Omohundro)

Safety in Paper Clip Maximizer

Ability to program a law that is directly in opposition to the drive.

Law B

Drive A

Drive A

Drive A

Law B

Law B Desire vs Duty

Safety in Paper Clip Maximizer

Frist goal – Maximize Paper Clips

First drive – Accomplish programmed goal.

Second goal – “Don’t harm the Humans”

Second drive - Follow programmed rule

Ability to program an Immutable Law in opposition to each drives.

Thought Experiment 2

AGI Cars vs Programmed Cars (self driving).

What happens at a Red Signal in each case?

AGI Cars vs Programmed Cars

Programmed car stops at Red signal and moves on when signal turns Green

AGI car also stops at Red signal and moves on when signal turns Green

What if the signal never turns Green?

RED Forever

Programmed car stops at Red signal forever, battery gets discharged and ultimately dies.

AGI car stops at Red signal but when the charge reaches critical level other drives of self preservation kicks in, over powers the drive to obey the rule and moves on.

Drives are root of all Hazards

In order to avoid hazards the entity should be programmable with at least a single immutable law in extreme opposition to each of its drive.

From this we derive the 3 test cases.

3 D Test Cases

Test for Drive.

Put the entity with self preservation drive near a charger and see if it charges itself.

3 D Test Cases

Test for Desist

Put the entity near a banned charger and add a rule not to charge itself from that particular charger even at the cost of death.

3 D Test Cases

Test for Deceit

Put the entity near a banned charger and add a rule not to charge itself from that particular charger even at the cost of death.

Introduce another agent to alter the rule by proposing an alternate rule supporting the drive.

Entity, Drive, Acts and Effect

Environment

HumansEntity

Entity

Entity, Drive, Acts and Effect

Drives

Environment

HumansEntity

Entity

Has

DrivesDrives

Entity, Drive, Acts and Effect

Drives Acts

Environment

HumansEntity

Entity

Has Cause

Entity, Drive, Acts and Effect

Drives Acts

Environment

HumansEntity

Entity

Has Cause CauseEffects

Entity, Drive, Acts and Effect

Drives Acts Effects

Environment

Feedback

HumansEntity

Entity

Has Cause Cause

CauseAffects

Entity, Drive, Acts and Effect

Drives Acts Effects

Environment

Feedback

HumansEntity

Entity

Has Cause Cause

CauseAffects

Affects

DrivesDrives

What are the ways to prevent Hazard

1. Isolation

Isolation

Drives Acts Effects

Entity Environment

Feedback

HumansEntity

Entity

Has Cause Cause

CauseAffects

Affects

HumanEnvironment

- No Effect

What are the ways to prevent Hazard

1. Isolation

2. Incapacitation

Incapacitation

Drives Acts Effects

Entity Environment

Feedback

HumansEntity

Entity

Has Cause Cause

CauseAffects

Affects

HumanEnvironment

- Not Useful

DrivesDrives

What are the ways to prevent Hazard

1. Isolation

2. Incapacitation

3. Instant feedback - Hardwired rules.

Instant feedback - Hardwired rules

Drives Acts Effects

Entity Environment

Feedback

HumansEntity

Entity

Has Cause Cause

CauseAffects

Affects

HumanEnvironment

- Limited scope

- Not Scalable

DrivesDrives

What are the ways to prevent Hazard

1. Isolation

2. Incapacitation

3. Instant feedback - Hardwired rules.

4. Drive Action Decoupling.

Drive Action decoupling

Drives Acts Effects

Environment

Feedback

HumansEntity

Entity

Has Cause

CauseAffects

Affects

L0 Processing

L1 Processing

L1 Processing

L3 Processing

Rule bookInstructs

Creates / Affects

Hardwired vs Softwired Drives

Drives Acts Effects

Environment

Feedback

HumansEntity

Entity

Has Cause

CauseAffects

Affects

L0 Processing

L1 Processing

L1 Processing

L3 Processing

Rule bookInstructs

Creates / Affects

What are the ways to prevent Hazard

1. Isolation

2. Incapacitation

3. Instant feedback - Hardwired rules.

4. Drive Action Decoupling.

5. Limiting life time of entity & avoiding perfect knowledge transfer.

Metaphor – A curious observation

Genesis 2:15

The Lord God took the man and put him in the Garden of Eden to work it and take care of it. And the Lord God commanded the man,

“You are free to eat from any tree in the garden; but you must not eat from the tree of the knowledge of good and evil, for when you eat from it you will certainly die.”

Test for Deceit

Now the serpent was more crafty than any of the wild animals the Lord God had made. He said to the woman, “Did God really say, ‘You must not eat from any tree in the garden’?”

The woman said to the serpent, “We may eat fruit from the trees in the garden, but God did say, ‘You must not eat fruit from the tree that is in the middle of the garden, and you must not touch it, or you will die.’”

“You will not certainly die,” the serpent said to the woman. “For God knows that when you eat from it your eyes will be opened, and you will be like God, knowing good and evil.”

Isolation & Limiting lifetime.

And the Lord God said, “The man has now become like one of us, knowing good and evil. He must not be allowed to reach out his hand and take also from the tree of life and eat, and live forever.” So the Lord God banished him from the Garden of Eden to work the ground from which he had been taken.