Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe.

Making Omelets Without Breaking Eggs:Adding Enterprise Features to Condor

Jason Stowe

condor_q

1.0 Community

2.0 Condor += Hooks

3.0 Conversations with Miron

4.0 Configuration w/Pipes

5.0 Progress to date

Condor Community

com·mu·ni·ty

A group of people living … under the same government.

A group of people having common interests: the scientific community

… Sharing, participation, and fellowship.

Community = People

Prior work Condor for movies: 1000+ CPUs (Linux/OSX)

70+ TB storage Active Community member

Currently:Cycle Computing

Condor Advice, Software & Enterprise Support

Windows/Linux/OSX

Now work on bigger projects with people I sat with @ CondorWeek2005…

Robert Nordlund, HartfordLife Cutting-edge commercial user

X000’s server VMs, X0000’s workstations

TBs storage,Vast IT resources

Commercial Users =sub-community

Share common interests… and resources

- Windows (&Linux/Solaris)- Commercial DB (Oracle)- Large Storage Capacity- Network Infrastructure- Available Tools:

JMS, Web clusters, DNS

How to Leverage Enterprise tools for Business Requirements -

Disaster Recovery/etc.

Bob Nordlund’s idea: Condor += Hooks

Configuration:Contribute hooks to Condor,

no workarounds (like extra daemons used in past)

How do we get this done?

How do we make an omelet,without breaking community eggs?

A Conversation with Miron

Funding

Text Here

Condor TeamWisconsin

Gov’t & Other Grant Funding

Sources

Text Here

Condor TeamWisconsin

Gov’t & Other Grant Funding

Sources

Government Research

AcademicInstitutions

Science & Research

Condor TeamWisconsin

Gov’t & Other Grant Funding

Sources

Government Research

AcademicInstitutions

Industry

Science & Research

Couldn’t industry provide funding for a team of people?

Objectivity/Independence

Condor is Attractive:Open, Innovative, Community

Problem with Large Enterprise Funding Contracts

= Dependence and Potential Conflict of Interest

Needs of the many must outweigh the needs of the few or the one.

What is the optimal way from Miron/Wisconsin’s perspective?

Proposed Solution

Condor TeamWisconsin

3rd Party Integrator HartfordLife

Enterprise

Wisconsin Responsible for:• Architectural Decisions – Final approval for features• Code Review/Integration• Release Planning• Maintenance

Proposed Solution

Condor TeamWisconsin

3rd Party Integrator HartfordLife

Enterprise

Integrator Responsible for:• Wisconsin – Build & Test, architecture feedback,

development, code review, testing, and integration• Enterprise – requirements, scheduling, & support

Enterprise funds feature addition

Proposed Solution

Condor TeamWisconsin

3rd Party Integrator HartfordLife

Enterprise

Lowers overhead by dealing with

integrator not many companies

Scalable, Schedule

ResponsibilitySupport

For the new feature:Wisconsin is Architect/Maintainer

Integrator is Developer/QA

Intellectual Property

Sharing, participation, and fellowship.

One Unified Condor

IP for Condor Code changes must be donated back to Condor

Plan the work. Work the plan.We got started…

HartfordLife = EnterpriseCycle = Integrator

Wisconsin = Governing/Architect

Bob and I called someone we sat next to

@CondorWeek 2005

Ian Chesal, AlteraWritten Innovative Middleware,

Priority Scheduling and Quill (yay!)

Share common interests…

Configuration w/Pipes (CWP)

CONDOR_CONFIG = cat file |

CONDOR_CONFIG = default_fileLOCAL_CONDOR_CONFIG =

getconfig http://address/$(hostname) |

Current Status

condor_status

vm1@Hartfordlife CLAIMED BUSYvm1@CycleComputing CLAIMED BUSYvm1@cs.wisc.edu CLAIMED BUSY

CWP Feature specification – DoneNMI Build & Test – Wrapping upCWP Implementation – In progressTesting & Code ReviewIntegration & Release

Condor_History

Community

Enterprise Community

Condor += Hooks

Model for feature addition

if you are a Condor user

Need support with new functionality

(Condor += Hooks)

Participate in the Community

Talk with Miron.Talk with me.

(Talk with everyone @CondorWeek 2006)

Help the community continue to work well for everyone.

Thank you. Questions?

jstowe@cyclecomputing.com