The .NET MF Codeplex Distribution HowTo

In November 2009, the .NET Micro Framework team at Microsoft released the 4.0 version of the .NET Micro Framework (.NET MF) as an open source product under the Apache 2.0 license. The 4.0 version included the .NET Micro Framework SDK and the Porting Kit.

Together with 4.0 we launched our community development Web Site and gathered important contributions. In order to boost the number of contribution, facilitate tracking any issues and create a stronger and more cohesive community development experience we are now publishing the 4.1 source code in this CodePlex project as well and allow a complete Community Development experience through Visual Studio and TFS. The license for this project is the same Apache 2.0 license of the .NET Micro Framework 4.0 version. As the .NET Micro Framework ships with the Open Source lwIP TCP/IP stack and the OpenSSL distribution as well, those licenses have been appended to the Apache 2.0 license. Details are on the home page of this project.

This project is experimental and Microsoft reserves the right to delete this project should we find out that it is not adequate to the goals above. In order to maintain the highest level of quality the .NET MF team will monitor the community to make sure the product are supported as needed by our customer.

 

Table of Contents

The CodePlex distribution, the Porting Kit and the SDK: who should use what and what for

This distribution is not intended to substitute the PK. The main objective of this project is to provide more visibility on the evolution of the .NET MF and to make community development efficient. The .NET MF team will continue to distribute and support the .NET MF SDK and PK. Users that need to build a managed application or device image from the samples in the SDK and PK should continue using the SDK and PK distributed through the Microsoft Downloads site.

The SDK installs the Visual Studio integration component as well as .NET MF assemblies. The SDK allows developing managed applications for the .NET MF emulator or a supported device. The Porting Kit (PK) includes all the sources of the Object Model, Runtime and Device Drivers, and a simplified build environment to build a device image. The PK allows porting the .NET MF to a SoC with a supported instruction set, currently ARM, Thumb, Thumb2, Blackfin, and SH2/A.  In general the following rules apply:

  • If you need to build a managed application for a supported device or the .NET MF emulator you should use the SDK only.
  • If you need to (a) build or modify the device drivers for a specific SoC, (b) develop interoperability components for a specific SoC, and (c) modify the runtime and/or object model, you should use the .NET PK and SDK .

The CodePlex project instead can be used to:

  • report an issue through the CodePlex Issue Tracker
  • download and use bug fixes and new features for the .NET MF SDK or PK before the next official version is even available: in this case you first check if the CodePlex distribution provides an Alpha or Beta release for the SDK and PK. If it does not you  should use the CodePlex web site to download the bug fixes you are interested in, and then apply them to the SDk and PK you are using.
  • contribute new components to the .NET MF code base: any component, with the exception of the VS integration module, can be developed in the SDK and/or PK. If you are unsure how to use this distribution, please use the SDK and/or PK to develop your components and then submit the changes through the CodePlex distribution. 
  • re-build the SDK and PK installers

The remainder of this document is a supplement reference on how to use the CodePlex distribution of the .NET MF. A required pre-reading in the PK documentation for the .NET MF 4.1 version that can be found on the Microsoft Downloads site.

The contents of the CodePlex distribution

Below you will find information about the following subjects:

-          The code base structure and permissions

-          The differences with the PK

-          The sources layout

-          How to use the PK downloads with the distribution

-          What you can build and how

-          How to use Issue tracker

Code branch structure

In order to better serve the interest of the community and allow selective planning and development the source code will be replicated on a number of branches. One branch (client_v4_1) will be used as a reference for the official version that shipped on the 24th of August as a QFE to the 4.1 version shipped in late July of the same year. The first child branch of client_v4_1 is the branch client_v4_2, from where the next official version (4.2) will be shipped. Only the .NET MF team will be able to modify sources in this branch. The client_v4_2_dev branch will be the working branch for the .NET MF team. The reverse integration to the official branch will be carried out by the .NET MF team for planned releases such as Previews, Alphas, Betas, RCs, and the RTM.

The client_v4_2_comm branch will host the community development. Integration and reverse integration to and from the client_v4_2_dev branch will be carried out by the .NET MF team on a planned schedule that will be posted on this site.

The members of the .NET MF Core Tech Team will always be allowed to check in the community branch and so the owners of a project for the time of the project. Whenever a community member proposes a new feature or a set of improvements, the project can be picked up by a member of the core Tech Team that will supervise its implementation by providing guidance and reviewing the design and the coding. The community members working on the project will be allowed access to the community branch for the duration of the project, including the necessary time to stabilize the code base. You can find a more detailed explanation about the community development process on the .NET Micro Framework web site.

 

Code Base Branches

Figure 1 - Source code branches

 

Differences between CodePlex and PK codebase

The PK and the CodePlex distribution are largely identical. As the PK is a subset of this distribution, the documentation of the PK is still the main reference.

To work with the CodePlex project, you will need to read and understand the Porting Kit documentation. The rest of this document assumes that you are familiar with the Porting kit and describes the build process as the differences from the documentation.

Over time we will make it easier to use this code base and eliminate the differences with the PK.

The CodePlex distribution of this project includes all the sources of the .NET MF code base, including the Visual Studio integration component, the project files for the installers, and the documentation.

In order to rebuild the SDK and PK one needs to install the Crypto package of the PK and, optionally, the TCP/IP Stack companion binaries for the RTIP stack for those device images that requires it.  

To understand the few differences between the CodePlex codebase and the PK we will look more deeply into three areas:

1)      Environment initialization scripts

2)      Project files and build tree selection

3)      PK generation configuration

 

Environment initialization scripts

The NETMF build system covers a number of processors and in some cases, multiple tool chains for those processors.  The specific build that you generate will be determined by the environment variable that are set up through initialization scripts.  It is important that you understand this initialization process so that you can set up the right environment.  The PK and Codeplex distribution contain a set of setenv_xxx.cmd initialization scripts in the %SPOCLIENT% directory. Those scripts are copied from the %SPOCLIENT%\tools\scripts\port directory during the build of the installer and differ from the actual codebase initialization scripts because the call the setenv_base.cmd script with the PORT parameter. The PORT parameter causes the setenv_base.cmd script to look for the environment variables established by the native compiler tool chains for the architecture of choice (e.g. ARM RVDS or Keil MDK, Renesas HEW, etc.). Conversely, the original initialization scripts rely on a fixed layout of the binaries and headers of those tool chains that is documented here. You can easily use the original scripts by copying over the tool chains binary in the required directory structure. The original tool chains layout is more suitable for integrating the development tools in the source control mechanism, which could suit a larger company with more employees that need to access the code base. In order to use the PK initialization scripts one can just change the setenv_XXX.cmd script of this distribution with the ones in the directory above. The setenv_base.cmd scripts will at the end create a set of environment variables to be consumed by the targets and settings files in %SPOCLIENT%\tools\target directory. Please note that the targets and settings file of the PK and CodePlex distribution are identical and all differences are driven by the environment initialization.  The PK-style initialization scripts will also set the PORT_BUILD environment variable which will modify the way some .dirproj files and .proj files are interpreted.  For example, when PORT_BUILD is set, it is not possible to build the SDK or PK installers.  Generally speaking the initialization scripts can be easily modified to suit one's needs and at the beginning we will require that one either re-creates the development tools layout documented in the above mentioned document, or modify the initialization scripts accordingly, e.g. by unsetting PORT_BUILD.  The most important environment variables set by the initialization scripts are

SPOROOT and SPOCLIENT for the .NET MF

for the ARM development tool chains (e.g. for the script setenv_RVDS4.0.cmd):

ARMBIN
ARMCONF
ARMDLL
ARMHOME
ARMINC
ARMLIB
ARMLMD_LICENSE_FILE
ARMROOT

for the HEW tool chain for SH (e.g. for the script setenv_shc.cmd):
SHC_LIB
SHC_INC
SHC_TMP
SHC_TOOLS_BIN

for the GCC GNU compiler (e.g. for the script setenv_gcc.cmd):
GNU_VERSION
ARMINC
ARMLIB
GNU_TOOLS
GNU_TOOLS_BIN
GNU_TARGET
COMPILER_PATH

Environment variables for controlling the build flavor

There are three environment variables you can use to control the build flavor.  You can set this variables before building, and make sure that you will always build, by default, the desired flavor of your binaries. 

FLAVOR_ARM=debug
FLAVOR_DAT=Debug
FLAVOR_WIN=Debug

These variables effectively complement adding the clause '/p:flavor=[debug|release]' to your build commands.

Project files and build tree selection

One other difference that the environment initialization creates is the build tree selection, i.e. how the build system will traverse the build tree to build the tools and the sources. The traversal is largely dominated by the environment variable PORT_BUILD. This environment variable is set by initialization scripts of the original PK and is not created by the other, causing a somewhat different behavior in choosing the tree traversal.

PK Generation and configuration

The PK generation is controlled by the scripts and resources contained under the directory %SPOCLIENT%\tools\script\port. Interesting the file SourceCopyList.txt that lists what files in the original distribution are substituted with equivalents in the PK. This reconnects back to the previous two points. Please note that some files are deleted as well (the ones with a ‘-‘ sign in front f them), as it is the case with the settings and target files for the Visual Basic support.

The sources layout

The layout of sources in the CodePlex distribution is the same as in the PK with the addition of the %SPOCLIENT%\ue directory which contains all the documentation. In addition the CodePlex codebase includes some binaries as well in the %SPOROOT% directory. Those binaries are used for the automation testing as well for building the installers.

How to use the PK downloads with the distribution

The satellite packages of the PK can be used to complement the CodePlex distribution with the Crypto libraries binaries and the RTIP TCP/IP and SSL stack. Crypto libraries are currently required to build the SDK or the emulator from the distribution while the second is optional. To get the Crypto Libraries you will need to install the Porting Kit first and then download and run the Crypto installer package.  After that you need to copy the %SPOCLIENT%\crypto binaries from the PK into the CodePlex distribution. The CodePlex distribution and the PK can be used on the same machine but there is no guarantee that they would not interfere. The SDK instead should never be installed on the same machine as the CodePlex project codebases.

What you can build and how

The CodePlex project builds mostly with the same rules as the PK, but one needs to decide whether he wants to use it as a PK or as a complete distribution. In the second case the original initialization scripts should be used and the crypto package should be installed and binaries move over to the CodePlex codebase. Also the native compiler tools of choice should be laid out following the guidelines in this document

NOTE: In order to build the SDK and PK installation one needs to build the documentation as well or modify the project file and Wix file to exclude the documentation.  To build the documentation one needs to install the latest Visual Studio SDK for Visual Studio 2008 and 2010 and then build the sdk.dirproj project for the release flavor. The documentation build can only be performed when the PORT_BUILD environment variable is not set.

Below you find the operation one needs to carry out to build the CodePlex distribution, step-by-step:

Execute once:

1)      Install TFS plug-in for Visual Studio 2010

2)      Connect to the “netmf” project on CodePlex and download the distribution

3)      Install the basic PK package and the Crypto libraries package

4)      Move the crypto libraries from the %SPOCLIENT%\Crypto\Lib directory of the PK to the same location in the CodePlex distribution (these libraries are not under CodePlex TFS source control because the licensing issues)

Build SDK or emulator:

5)      Open a command line shell (cmd.exe) and change current directory to one branch

6)      Initialize the environment with the setenv_xxx.cmd script for the native tool chain of your choice.  If you choose to use the porting kit style initialization scripts you might have to unset the PORT_BUILD environment variable.

7)      Issue the command “msbuild [/p:flavor=debug|release] sdk.dirproj” to rebuild the SDK and the emulator. After the build is complete you will find the SDK installer under %SPOCLIENT%\BuildOutput\public\Debug\Server\msm

Build a solution for a device image:

8)      Issue the command “msbuild sdk.dirproj /p:flavor=[debug|release] /p:MajorVersion=5 /p:MinorVersion=3 /p:BuildNumber=45 /p:RevisionNumber=66 /p:SignBuild=true” as in the PK. Please note that you can impose the complete build version number and therefore replicate an older one if you need to.  Also refer back to setting the FLAVOR_XXX environment variables for controlling that the build flavor is consistent for all types of binaries.

To use the SDK from within the distribution command line:

1)      Start a command line shell (cmd.exe) as Administrator mode

2)      Initialize the environment with the setenv_xxx.cmd script for the naïve tool chain of your choice

3)      Issue the command “msbuild [/p:flavor=debug|release] sdk.dirproj /t:prepVS” to generate and run the pseudo-installer script “%SPOCLIENT%\tools\script\mfpseudoinsaller.cmd”. You can now use an initialized command line environment to open C# project files with Visual Studio. E.g. you can issue the command “devenv.exe Test\Platform\Tests\CLR\Microsoft.SPOT.Graphics\Bitmap.csproj” to run the bitmap tests from Visual Studio using the binaries in %SPOCLIENT%\BuildOutput\public\[debug|release]”

4)      Uninstall the pseudo-SDK with “%SPOCLIENT%\tools\script\mfpseudoinsaller.cmd /u” when you no longer need it

To generate the PK installer:

1)      Issue the command “generate_port <dst> [debug|release|instrumented|RTM] copy_only zip BuildNumber=1234 RevisionNumber=5678”. You can use ‘copy_only’ if you already built the SDK for all flavors you are indicating in command above. The release flavor is needed even when not indicated. If you omit copy_only the script should build all the required flavors autonomously. This scripts generated a compressed version of the PK along with an uncompressed one.

2)      Use the utility PartitionPK.exe to create the PK installer. An example of command line is provided below. Please refer to the help of the tool for further instructions. Issue the command “PartitionPK.exe PKProduct\PKProduct.wxs MicroFrameworkPK <dst> BuildFlavor=;BuildTreeServer=%SPOCLIENT%\BuildOutput\public\Release\server;BuildTreeClient=%SPOCLIENT%\BuildOutput\public\Release\client;BuildTreeClientTest=%SPOCLIENT%\BuildOutput\public\Release\test\client;BuildTreeServerTest=%SPOCLIENT%\BuildOutput\public\Release\test\server;WixSpoClient=%SPOCLIENT%;ProdVerMajor=4;ProdVerMinor=1;ProdVerBuild=69216;ProdVerRevision=0;ProdVer=4.1.69216.0;ProdVerMajorMinor=4.1;MSBuildProjectDirectory="%SPOCLIENT%";MSBuildProjectFile=client_v4_1.branchproj;MSBuildProjectExtension=.branchproj;MSBuildProjectFullPath="%SPOCLIENT%\client_v4_1.branchproj";MSBuildProjectName=client_v4_1;MSBuildBinPath="C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319";MSBuildProjectDefaultTargets=Build;MSBuildExtensionsPath="C:\Program Files\MSBuild";DesktopTargetFrameworkVersion=2.0.50727;VisualStudioIntegrationDir="C:\Program Files\Microsoft Visual Studio 2010 SDK\VisualStudioIntegration" NONE ARM=<dst>\Template3.1\client_v4_1. This command builds the Porting Kit installer for the basic PK

How to use Issue tracker

CodePlex allows for a fairly efficient issue tracking. This mechanism supersedes the “Support, Queries and Bug Reporting” forum on the .NET MF website. It is still appropriate and encouraged to use the forum, but every community member will want or will be required to file an issue to get a bug fixed or a DCR (Design Change Request) accepted. If unsure whether it is a bug, please use the forum where other users can help you finding a workaround or using correctly a particular API or type. Over time, we might move all forums issue to the CodePlex tracking system. In the picture below you find a snapshot of the classification keywords. We will discuss here how you should use them to get support from the Community.

 Issue Tracker

Figure 2 - Issue tracker fields

 

Status

All issues should be logged with a Status value of “Proposed” for a DCR and “Active” for bugs. When a community member fixes an issue you will find a change list number in the issue to refer to in the codebase.  

Impact

Use the Impact field with caution. Generally speaking, an issue has a “Low” impact if a workaround exists, a “Medium” impact if a workaround exists but involves recompiling the SDK or PK or is rather inefficient, and a “High” impact if there is no workaround.

Release

If you know that an issue impacts more than one release, always assign the release to the latest release available. It will be the care of the NETMF team to back-port the fix to all relevant releases when needed. If unsure, assign the issue to the release where you found the issue.

Assigned To

Always use the value “Microsoft”, it will be our care to redistribute the issue. If you are working on a community project then you know who to assign the issue to.

Component

The components are:

.NET MF SDK (with version): for any issue that regards the OM and/or the emulator

VS : for any issue that regards the Visual Studio integration, including building managed components

CLR: for any issue that regards the runtime (GC, Type System, Interop libraries, …)

.NET MF PK (with version): for any driver issue or build issue

Tools: for any issue with MFDeploy.exe, SolutionWizard.exe and other satellite tools

CodePlex build system: for any issue that regards the CodePlex codebase build or distribution that does not shows up in any of the above categories

 Microsoft will triage issue and answer them on a bi-weekly basis. You can expect your issue to be resolved at most in one month from the moment you log it.

Last edited Jan 10, 2012 at 8:39 PM by lorenzte, version 7