Developing NuGet
Post on 10-May-2015
605 Views
Preview:
DESCRIPTION
Transcript
1
Developing NuGetJEFF HANDLEY | DEVELOPMENT LEAD | NUGET | MICROSOFT
@JEFFHANDLEYJEFFHANDLEY.COM
Source: www.imdb.com 2
Airborne (1993)
3
4
5
6
NuGet LandscapeWHAT MAKES UP THE NUGET PROJECT
7
NuGet Project Open Source (owned by Outercurve Foundation)
◦ Releases shipped by Microsoft
Included in all Visual Studio installations
Millions of users (some knowingly, some not) around the world
Dozens of high-profile dependent projects at Microsoft
International contributors◦ Only a few external contributors are in the US
Very large backlog of issues
Conflicting requirements
Backward compatibility
Mixture of hard and soft deadlines
Extremely costly test passes
8
Outercurve and MicrosoftNUGET CLIENT
Outercurve’s NuGet Project
Microsoft’s NuGet-Based Package
Manager
In the Box with Visual Studio
NUGET GALLERY
Outercurve’s NuGet Gallery Projectwww.nuget.org
Azure Hosting provided by
Microsoft
Operated by Microsoft on behalf
of Outercurve
9
NuGet’s Core TeamNUGET CLIENT
Visual Studio Extension
WebMatrix Extension
NuGet.exe Command-Line
NuGet Packages
3 Developers
2 Testers
NUGET GALLERY
www.nuget.org
staging.nuget.org
docs.nuget.org
build.nuget.org
3 Developers
2 Testers
Always 1 member on ops/support rotation
10
NuGet’s Core TeamNUGET CLIENT
Visual Studio Extension
WebMatrix Extension
NuGet.exe Command-Line
NuGet Packages
3 Developers
2 Testers
NUGET GALLERY
www.nuget.org
staging.nuget.org
docs.nuget.org
build.nuget.org
3 Developers
2 Testers
Always 1 member on ops/support rotation
11
Release ManagementNUGET CLIENT
12
NuGet Releases
New release every 10 weeks since NuGet 1.5
2/12/20111.1
3/30/20111.2
4/25/20111.3
6/17/20111.48/30/2011
1.5
12/13/20111.6 4/4/2012
1.7
5/23/20121.8 6/19/2012
2.0
10/4/20122.1 12/12/2012
2.2
2/15/20132.2.1 4/25/2013
2.5
6/26/20132.6 8/22/2013
2.7
10/7/20132.7.1
1/14/20111.0
13
NuGet in Visual Studio Releases
2/12/20111.1
3/30/20111.2
4/25/20111.3
6/17/20111.48/30/2011
1.5
12/13/20111.6
Introducedwith
ASP.NETMVC 3and
Web Pages v1
4/4/20121.7
5/23/20121.8 6/19/2012
2.0
10/4/20122.1 12/12/2012
2.2
2/15/20132.2.1 4/25/2013
2.5
6/26/20132.6
8/22/2012Visual Studio2012 RTM
4/4/2013Visual Studio
2012 Update 26/26/2013Visual Studio2013 Preview
9/9/2013Visual Studio
2013 RC
1/14/20111.0
5/31/2012Visual Studio
2012 RC
2/28/2012Visual Studio2012 Beta
8/22/20132.7
10/7/20132.7.1
Visual Studio2013 RTM
14
Release Tiers CI build from Outercurve Foundation always available at http://build.nuget.org
◦ Usually 100 or fewer users◦ Build created from every commit
Beta/RC builds from Microsoft occasionally available at http://nuget.codeplex.com◦ 1,000 – 2,500 users◦ Published after we hit code-complete◦ Real-world testing of newly introduced features or significant design changes
Final builds from Microsoft published to the Visual Studio Extension Gallery◦ 500,000 – 750,000 users◦ Released after our test team signs off, a few weeks before Visual Studio locks down for its next release◦ Last stage real-world testing from users that are knowingly using NuGet and are a bit more forgiving
In the box with Visual Studio◦ Millions of users◦ Many of these users don’t know what NuGet is and will file Visual Studio bugs if we break them
15
Backlog CodePlex: 858 open issues as of 10/4/2013
Microsoft Internal Issues◦ TFS: Fluctuates from 0 to 25
◦ Dr. Watson reports◦ Microsoft Connect◦ Microsoft shipping requirements◦ Future Visual Studio release issues
Strategic Product Features◦ Schedule alignment for release within Visual Studio and Visual Studio Update
16
Strategic Product Features Package Manager for the Microsoft Development Platform
◦ Not just ASP.NET◦ Not just .NET◦ Includes C++, Windows Store, and Windows Phone◦ Cross-platform support with Command-Line, Xamarin Studio, MonoDevelop, and SharpDevelop
Improving Package Discovery (through collaboration with the NuGet Gallery crew)
Avoiding Package Version Hell
Platform Multi-Targeting and Re-Targeting
New Usage Workflows for Improved Application Maintenance and Ecosystem Growth
Helping Improve Gallery Reliability
17
Triaging Issues1. Is it a high-priority issue (crash, hang, regression)?
◦ Include it in the current release
2. Is the bug fix or feature needed for the current release?◦ Include it in the current release
3. Is it a high value item that fits in the next release?◦ Include it in the next release
4. Do we agree with the concept?◦ Put it in the “Up For Grabs” release◦ Ideally picked up by the community◦ High-voted items are reviewed when planning releases and some items are pulled in
5. Do we disagree with the concept?◦ Close it
18
Priorities1. Strategic roadmap that integrates with the Microsoft development platform
◦ Features planned for 2-3 releases at a time◦ Align with Visual Studio releases: Preview, Beta, RC, RTM, Visual Studio Update
2. Community-reported high-value bug fixes or features◦ Including those that come with external pull requests
3. Highly-voted community feature requests
4. Community-reported high-value bug fixes
5. Microsoft partner team bug reports
6. Microsoft partner team feature requests
7. Community-reported low-value bug fixes
19
Philosophy Small and easy bug fixes are low priority
◦ Otherwise, we’d spend 100% of our time making small changes that affect only a handful of users
Don’t prematurely act upon cool ideas◦ Does the idea spark new strategic concepts?◦ Do other new ideas relate?◦ Can the feature be implemented with backwards compatibility?◦ Could this feature become obsolete based on other potential strategic directions?
Favor community-requested features over partner team features◦ Avoids scenario-specific features that conflict with broad usage◦ Encourages partner teams to find workarounds with existing features◦ Those workarounds often surface pain points others might also be experiencing
We’d rather delight 1,000,000 external users than gain 1 more internal partner team
20
Conflicting Requirements Microsoft’s Customers vs. NuGet’s Ecosystem
◦ Group Policy control over package sources
Bold New Features vs. Backwards Compatibility
◦ Package Restore◦ Build-time reference resolution
NuGet User Experience vs. Microsoft’s shipping requirements
◦ Online consent◦ Loading only Microsoft-signed assemblies◦ Modified content files
Favor user desires as much as possible
Work within the constraints of Microsoft’s shipping requirements
Negotiate and compromise with privacy, security, and legal representatives to arrive at acceptable implementations
21
Source ControlNUGET CLIENT
22
master
mer
ge
FeatureA
FeatureB
Branch: ͞S2.7͞T
Branch: ͞S2.7.1͞T
mer
gem
erge
Branch: ͞S2.8͞T
Tag: ͞S2.7͞T
Tag: ͞S2.7.1͞T
Git
Branch per feature Branch per version Tag for release
23
Continuous Integration Outercurve Builds
◦ TeamCity – running on Virtual Machines in Windows Azure◦ 1 VM build agent with .NET 4.0 for building the VS Extension for VS 2010 and VS 2012◦ 1 VM build agent with .NET 4.5 for building the VS Extension for VS 2013
◦ Hosted at http://build.nuget.org◦ Produces non-localized, unsigned builds with Outercurve branding and licensing◦ “Developer Branches” build configuration builds all “dev-” prefixed branches
Microsoft Builds◦ TeamCity – running on Virtual Machines in a group lab
◦ 1 VM build agent with .NET 4.0 for building the VS Extension for VS 2010 and VS 2012◦ 1 VM build agent with .NET 4.5 for building the VS Extension for VS 2013
◦ Only accessible on the corporate network◦ Produces localized, signed builds with Microsoft branding and licensing
24
Going Dark Microsoft-confidential work cannot go to CodePlex
◦ Integration with new versions of Visual Studio◦ Support for new platforms (e.g. Windows Phone 8)
Team Foundation Service private project◦ Fork the NuGet Git repo into that project◦ Move confidential development into that fork◦ Keep all other development on CodePlex
Merge commits into TFS daily
Configure our Internal Builds to use the TFS fork instead of CodePlex
25
TestingNUGET CLIENT
26
Release Timing Release sign-off takes about 4 weeks after code-complete
◦ 40% of our 10-week release cadence is spent testing and fixing bugs
Snap releases to the Visual Studio schedule
Release to the gallery first, and then integrate into Visual Studio’s build
Between Visual Studio Beta and RTM, we have 2 or 3 tight quality-driven releases◦ Meet Microsoft’s release requirements◦ Address any partner team blocking issues◦ Focus on extremely broad use of the product
After Visual Studio hits RTM, we go big with features and aim for a 3-month release
27
Test Matrix Dimensions Visual Studio SKUS
◦ VS 2010◦ Express for Web◦ Express for Phone◦ Pro+
◦ VS 2012+ - All SKUs
Project Types◦ Virtually all Microsoft Project Types
◦ Lightswitch◦ Windows Store◦ Windows Phone◦ Portable Class Libraries
Languages: C#, VB, F#, JavaScript
Operating Systems◦ Windows XP◦ Windows Server 2003+◦ Windows Vista and Windows 7◦ Windows 8◦ Windows 8.1
Different types of NuGet Packages
Different types of NuGet Package Sources◦ Gallery◦ NuGet.Server◦ Local Disk◦ UNC Share
Source Control Integration◦ TFS◦ Git, Mercurial◦ None
NuGet Clients◦ VS Extension◦ WebMatrix Extension◦ NuGet.exe Command-Line
Plus, all of the combinations of actual features
28
Achieving Release Sign-Off Unit Tests managed and run by developers
End-to-end Functional tests managed and run by developers and through automation◦ Based on a PowerShell test system included within our project◦ Runs within Visual Studio, automating VS through the DTE
QA Tests for interactive testing◦ Mostly automated, but exploratory testing always finds more bugs◦ Run across a farm of machines in our group’s test lab, covering the machine configuration matrix◦ A representative set of combinations give us confidence◦ Uses an internal VS automation library to simulate user actions in VS
Group-wide bug bashes
Company-wide partner team bug bashes
Public Beta testing from CodePlex builds
Reach 0 bugs in CodePlex for the current release
29
Throughput109 Weeks•Total time working on those 10 releases
10 Releases•816 work items have been completed
3 Developers•Averaging 27 work items per release
2 Testers•Each testing 3.7 bug fixes or features per week
30
Success Never defined by throughput
No regressions
No minor updates necessary
Users delighted by new features
Took steps toward our strategic roadmap
Addressed highly-voted community bugs and features
Accepted a level of failure in smaller user base releases
31
Success Rate NuGet 1.6
◦ Required a 1.6.1 update◦ Regressions◦ Missed bugs in new features
NuGet 1.7◦ Success
NuGet 1.8◦ Success
NuGet 2.0◦ Success
NuGet 2.1◦ Success
NuGet 2.2◦ Success
NuGet 2.5◦ Success
NuGet 2.6◦ Success
NuGet 2.7◦ Required a 2.7.1 update◦ Regression◦ Missed bugs in new features
32
Tiered Releases are Key
Our tiered release approach worked for 2.7
2.7.1 update in place in time for integrating into Visual Studio
Thousands of users affected instead of millions
As Visual Studio’s broad user base jumps from NuGet 2.2 to NuGet 2.7, they’ll have no idea we ever introduced regressions
You wouldn’t have noticed the camera in Airborne if I hadn’t pointed it out to you
33
Q & A
Get a NuGet Sticker!
Twitter: @jeffhandley
Website: jeffhandley.com
top related