Pharo sources in git

Pharo Sources in Git: a Strawman

Dale Henrichs Dr. Metacello Nov. 2, 2016

Pharo Kernel Image• 1 git repo for all kernel packages

• all packages same version

• tags for releases

• BaselineOfPharoKernel

• developers use it to update to latest kernel version

• and manage their own PharoKernel clone

• bugfixes and private extensions

Pharo Dev Image• Assume that pharo dev image is built upon the minimal core

• Multiple git repos: 1 per independent functional unit

• one BaselineOfPharoDev that collects together the component projects used to construct the Pharo Dev Image

• PharoKernel project referenced by tag pattern: `v7.?` which matches all tags in PharoKernel repository starting with `7.`

• Similar tag name schemes used for other component projects

• maintain Pharo fork of projects that are not owned by Pharo team:

• Metacello, FileTree, etc.

Pharo kernel [60d52c5]

Git repository structure at image build time

[SHA]

ImageGitHub

Pharo kernel

Pharo dev

Metacello

git-cache

Pharo dev [f5e8170]

Metacello [48f1da1]

W E B

Freshly downloaded image needs to be properly connected to project

repos for development

ImageGitHub

Pharo kernel

Pharo dev

Metacello

git-cache

W E B

?• similar problem exists today — Monticello packages

reference the Pharo repo (where package loaded from) not the original development repo

ImagePharo kernel [60d52c5]

Pharo dev [f5e8170]

Metacello [48f1da1]

Metacello records SHA of commit at load time and current repo, but not original

download repo

GitHub

Pharo kernel

Pharo dev

Metacello

W E B

Pharo kernel [60d52c5]

git-cache

Pharo dev [f5e8170]

Metacello [48f1da1]

Connecting to proper repo involves executing the following chunks of code

clone/lock/rehome (roughly based on tODE implementation)

local image connection complete

ImagePharo kernel [60d52c5]

GitHub

Pharo kernel

Pharo dev

Metacello

…/shared

Pharo kernel [1f3f58a]

W E B

NOTE: image SHA doesn’t match cloned SHA

ImagePharo kernel [60d52c5]

GitHub

Pharo kernel

Pharo dev

Metacello

…/shared

Pharo kernel [1f3f58a]

W E B

• ‘version skew’ is not in and of itself a problem … but tools are needed to inform user of the issue

• image state is preserved correctly — visual reminder that image SHA does not match clone SHA

• once the repo is connected

• the developer can checkout the SHA that matches image

• or, load the latest version …

Common Occurrence

tODE Project List, showing version skew

• tool makes it obvious that there is a potential problem

tODE Project List menu

• `skew diff` and `skew save` menu items support for handling version skew

How does the clone tool know what to clone?

(Metacello only records SHA)

• in tODE I use a project entry to specify what to clone and well-known repository locations: `shared` and `image` to specify where to clone

In tODE, project enries served up by GsDevKit_home `gh-pages`

For Pharo: MetacelloProjectLoadSpec

• combo of tODE project entry, SmalltalkCI SCIMetacelloLoadSpec and CatalogProject

• MetacelloProjectLoadSpec served up by catalog.pharo.org or a Pharo/Squeak/GemStone oriented server or ???

MetacelloProjectLoadSpec instance shipped in Pharo image

ImagePharo kernel [60d52c5]

…/shared

loadSpec[ ‘PharoKernel’, ‘git://github.com/pharo-core/PharoKernel:v7.?/src', … ] GitHub

Pharo kernel

Pharo dev

Metacello

W E B

• reference to primary project repo

clone/lock/rehome operation performed using MetacelloProjectLoadSpec

(repo location updated to reflect clone location)

ImagePharo kernel [60d52c5]

GitHub

Pharo kernel

Pharo dev

Metacello

…/shared

loadSpec[ ‘PharoKernel’, ‘filetree://.../shared/PharoKernel/src', … ]

Pharo kernel [1f3f58a]

W E B

How do multiple images share repos (load specs)?

ImageGitHub

Pharo kernel

Pharo dev

Metacello

…/shared

Image Image

? ?Pharo kernel

W E B

loadSpec[]

How do multiple images share repos (load specs)?

ImageGitHub

Pharo kernel

Pharo dev

Metacello

…/shared

Image Image

Pharo kernel

W E B

Pharo kernel.ston

…/projects

loadSpec[] loadSpec[]loadSpec[]

• load spec STON file written to disk in /…/projects dir

• disk version of load spec object shared by all images

How to manage forked projects

ImagePharo kernel [60d52c5]

GitHub

Pharo kernel

Pharo dev

Metacello

Pharo fork

loadSpec[ ‘PharoKernel’, ‘git://github.com/pharo-core/PharoKernel:v7.?/src', … ]

GitHub/YOU

W E B

…/shared…/projects

How to manage forked projects —edit load spec to reference fork—

—save to disk—

ImagePharo kernel [60d52c5]

GitHub

Pharo kernel

Pharo dev

Metacello

Pharo fork

loadSpec[ ‘PharoKernel’, ‘git://github.com/YOU/PharoFork:master/src', … ]

GitHub/YOU

W E B

…/shared

Pharo fork.ston

…/projects

How to manage forked projects —clone/lock/rehome/share—

ImagePharo kernel [60d52c5]

GitHub

Pharo kernel

Pharo dev

Metacello

Pharo fork

loadSpec[ ‘PharoKernel’, ‘filetree://.../shared/PharoFork/src', … ]

GitHub/YOU

…/shared

Pharo fork [85c552c]

W E B

Pharo fork.ston

…/projects

Pharo kernel [1f3f58a]

Private Image Clone —edit image load spec—

—don’t save to disk—

ImagePharo kernel [60d52c5]

GitHub

Pharo kernel

Pharo dev

Metacello

Pharo fork

loadSpec[ ‘PharoKernel’, ‘filetree://.../shared/PharoKernel/src', … ]

GitHub/YOU

…/shared

Pharo fork [85c552c]Pharo fork.ston

…/projects

W E B

Full Picture

Image

…/shared

Image Image

Pharo fork

Pharo fork.ston

…/projects

Metacello.ston

Pharo dev.ston

loadSpec[] loadSpec[]loadSpec[]

catalog.pharo.org

GitHub

Pharo kernel

Pharo dev

Metacello

Pharo fork

GitHub/YOUPharo

kernel.stonPharo kernel

W E B

General Caveats: Don’ts

• git submodules and git subtrees are not as flexible as Metacello project dependencies, so they are not needed

• Don’t put different platforms/releases on different branches

• FileTree is the anti-pattern and I wouldn’t do that again

• requires ‘cherry-picking’ to share common code between branches

• use platform-specific packages in concert with platform-common packages so that common and platform code can be kept in sync common changes

General Caveats: Dos

• Do use continuous release model as much as possible

• as long as there are no API breaking changes latest commit on `master` branch is the RELEASE

• used by Metacello where I have by design avoided API breaking changes for ~7 years

• Use tags based on Semantic Versioning when API breaking changes cannot be avoided

• use the tag pattern matching in Metacello repository URL for referencing range of versions

• could easily introduce new syntax in Metacello `git:` URL that specifies ranges of versions

• clone of git repo using a tag leaves the clone without a branch —commit disallowed until branch assigned