FreeBSD: The Next 10 Years (MeetBSD 2014)

FreeBSD: The next 10 years

Jordan HubbardMeetBSD 2014

But firstA look back at where we came from!

FreeBSD 1.0

From [email protected] Tue Nov 2 14:51:38 1993Path: sran230!sranhd!sranha!wnoc-tyo-news!nec-tyo!nec-gw!sgiblab!spool.mu.edu!agate!agate!usenetFrom: [email protected] (Jordan K. Hubbard)Newsgroups: comp.os.386bsd.announceSubject: FreeBSD 1.0 RELEASE now availableFollowup-To: posterDate: 1 Nov 1993 16:12:20 -0800Organization: Lotus Development IrelandLines: 61Sender: [email protected]: [email protected]: <[email protected]>NNTP-Posting-Host: agate.berkeley.edu

The first "official" release of FreeBSD 1.0 is now available, no moregreek letters - this is the "production" release ...

Happy 21st Anniversary!

FreeBSD will be 21 years old tomorrow(and can drink legally)

Our first distribution media

(OK, I’m lying)

FreeBSD 1.0

Our actual distribution media

1.2 MB

FreeBSD 1.0

Some of the consequences...

The 1.0 ports collection

My first FreeBSD dream build machine...

I paid $1600 for my first 1Gb hard drive to do builds...

My first FreeBSD dream laptop

<= WarnerLosh

Windows->

First conference (1999)

Commercial interests20 years ago

• Internet Service Providers (large and small)

• SOHO web servers / routers (very DIY)

• Major services like Yahoo!, Hotmail, etc.

• Basic developer desktop machines

• Majority of FreeBSD machines were physical PCs that ran off of AC power

Key take-away: FreeBSD’s role was fairly overt

20 years later...

New install media: 64GB USB thumb drive. Holy crap!

20 years later...

RB-Pi: SBC Far more powerful than my first FreeBSD machine. Cost: Under $50 (with storage card)

20 years later...

My dream laptop has evolved (BSD instead of Windows)

20 years later...• FreeBSD release version is up to 10.1

• Over 24000 ports (vs 70!)

• Hundreds of committers, from both academia and commercial backgrounds

• Sources of long-term funding (Foundation, other sponsors)

• … But the commercial interests and overall market have changed significantly

Hey BSD: Let’s see that license again!

The GPLv3: Scaring the crap out of lawyers since 2007

The GPL experience

Commercial interestsToday

• FreeBSD is the underlying OS technology for routers, load balancers, security monitors, file servers, etc.

• Basis for “software appliances” like pfSense, FreeNAS and quite a few others

• Even the base OS for a very popular gaming console

• “Embedded” market is morphing and exploding

Key take-away: FreeBSD’s role, and even its name, is now fairly covert

Biggest Unix deployments now

(wat?)

Even in the Enterprise…• “BYOD” movement has slowed desktop growth

and fundamentally changed IT department mission

• “Cloud” software / storage / computational resource consolidation have all but killed the DIY datacenter

• Much better automation choices (*stack, chef, puppet) gives rise to the “2 guys with their laptops in a coffee shop” startup model

The rise of virtualization

• VMWare, Xenserver, Hyper-v, bhyve - all acceptable hypervisors. Majority of OS deployments (and devices) are no longer physical

• Automation tools work hardest to erase the notion of machine personality (and put it elsewhere)

• Being virtual and/or automated means the entire environment becoming far less human-centric

What is an Operating System?

A set of lego blocks

What do OS designers like to build?

Things they think are cool

We have to change our definition of what is cool

(Building blocks are cool)

More about mobile

• More Unix machines are running on batteries than are plugged into AC by several orders of magnitude

• Talking to one or more radios has replaced physical cables

• The obvious: Power consumption and dynamic interface / connection management is important

• Less obvious: High-level debugging and telemetry technology become critical to success

Ubuntu’s changing face

What does this all mean?

• We need to be open to fundamentally new approaches and ruthlessly cull what is no longer demonstrably useful to the 99%

• We need to be willing to shamelessly steal^H^H^H^H adopt things that are working well for others

• We need to take on some big-picture challenges that will appeal to the next generation of hackers (where’s the next mountain?)

One Project IdeaAll OS / App configuration data

• We need a lingua franca format yesterday!

• Whether it be XML, YAML or JSON, we also need a single API to read / write / abstract away the details of finding config data.

• Existing tools /services need to convert their legacy formats into this one and use the same API or the tower of babel will persist

Working Example

• All OS and app configuration data in OS X and iOS are XML plist files, even GNU emacs and X11.org’s preferences!

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>SULastCheckTime</key> <date>2014-02-09T02:45:56Z</date> <key>cache_fonts</key> <true/> <key>done_xinit_check</key> <true/> <key>no_auth</key> <false/> <key>nolisten_tcp</key> <true/> </dict> </plist>

One Project IdeaA centralized event notification system

• You just can’t deal with:

• Radios powering up and down at odd times

• Network configuration changing rapidly

• A node changing its identity almost entirely on the fly

• Critical system caches needing invalidation

… without a centralized way of being able to know about these sorts of events!

• notify(3) APIs in OS X / iOS and corresponding notifyd daemon

• Send it a SIGUSR1 on OS X and you will see a huge number of notification names and their subscribers in /var/run/notifyd_pid.status - it has become insanely useful!

"com.apple.system.lowdiskspace.system" uid=0 gid=0 333 17 32 port

"com.apple.system.timezone" uid=0 gid=0 333 slot 7 = 1 1018 1 memory 1011 10 port 1011 8 memory 1011 6 port

"com.apple.system.powermanagement.SystemLoadAdvisory" uid=0 gid=0 333 slot 13 = 19 406 44 port 944 12 port 406 35 memory

"com.apple.system.config.network_change" uid=0 gid=0 333 slot 39 = 129 190 17 memory 845 28 memory 845 26 memory

"ids-device-nearby-0FC85E3A-7779-4EAA-AD02-70292707A33C" uid=0 gid=0 333 229 30 port 402 19 port 257 30 port 242 23 port

Working Example

One Project IdeaService startup and wrangling

• /etc/rc.d is quite sophisticated for what it does, but the paint on /etc/rc is obvious

• Too many things need to know explicitly about dependencies (when can I start? What has to start before me?)

• Power wrangling and automation depend on being able to start and stop services easily and at will

Working Example

• I’m trying really hard not to suggest launchd here (so I won’t)

• The idea of registering everything up-front with a broker and then letting IPC / timers / HW events start things from there (in cascade fashion) is still the right architecture

• Even the linux die-hards have essentially grasped the necessity of systemd (even though they’re going to hate on it for awhile longer)

One Project IdeaTelemetry & Remote Debugging

• We really need a centralized way of being able to collect data from appliances / mobile devices

• The NSA don’t need our help (they already do this upstream) but FreeBSD downstream end-users DO

• Debugging the software stack on mobile devices is hard. It’s time for remote debugging support

One Project IdeaReference phone / tablet ports

• Running on SBCs is excellent for bootstrapping, but there’s no “real world” hardware to support (displays, radios, accelerometers, etc)

• Running on hardware you can actually use and carry around validates power / telemetry work

• Real Hardware forces you to think about the entire software stack

In summary

• We need to become more lego-like (and toss those legos we don’t need out of the box) in our architecture

• The hardware platforms we choose need to be genuinely relevant in terms of mass appeal

• We need to be more willing to learn from those who have gone before us in these emerging markets (BSD has a bit of a “rep” there)

FreeBSD 20th Anniversary Party

FreeBSD: The future is ours if we want it!