Development Environment

Now that we have Source Control all set up it is time to prepare our development environment.

NODE

Please read this entire section before you jump in and install NODE!

Node is a JavaScript runtime that uses Chrome's V8 engine. Because we are intending on building a cross-platform solution in JavaScript we need a runtime or hosting environment for our development efforts. Node is a perfect choice. It is the preferred choice of most JavaScript open source projects and has a monumental open source library of modules and plug ins that can be used in our project.

In order to promote as much common cross-functionality as possible I am trying to find common ground between all the associated pieces of the puzzle. The Chrome V8 engine is used in the Chrome browser as well as Node, Cordova (for Android), Electron and a host of other tools such as unit test runners. More about this later.

Node features the Node Package Manager commonly referred to as NPM. NPM is installed automatically with Node so there is no need to install it separately. In fact, each version of Node carries a specific version of NPM so it is wise to let Node manage this automatically.

NPM is used to download and manage JavaScript packages or libraries for your project. It saves package information in a special file in the root of your project called package.json. This file is a JSON data file. The cool thing about this system is that you include the package.json file in your source repository but you do not include the file folder node_modules where all the downloaded packages live. When you git a project from a source code repository you must run 'npm install' before you begin development. This invokes the npm package manger that reads the package.json file and automatically downloads and updates the packages in the projects node_modules file folder.

Node is a great system, however it has had it's share of challenges along the way. Earlier versions (3 and earlier) used a nested package system when storing the packages in the node_modules folder. This was problematic on windows machines as it created extremely long paths to dependent packages. This was changed in version 4 to a flat style system where all packages are stored in the root of the node_modules folder. In my opinion this is a much better solution and the Aurelia project depends on this so we will be using Node version 4 or higher in this project.

Node also had some issues with the way the package.json file managed the installed versions of packages. I am not going to dive into specifics but you can read more about it here. To solve this problem, newer versions of Node also include a file called package-lock.json. This file is used together with the package.json file to better resolve package versions in your project. This file should also be included your source control check in.

Before you install Node on your computer you should consider using Node Version Manager (NVM).
NVM is installed on your computer and you use it from a command line to manage the version of Node installed on a folder by folder basis. This means you can have different versions of Node installed for each project you work on. In my opinion this is a game changer. It is very easy to get caught up in the node package version vortex ... spending countless hours trying to resolve conflicts so you can run your project. I know from experience!

Please use NVM ... you will thank me later!

We will be using the latest LTS version of Node for this project so lets go ahead and install it now.

1. Install NVM following the instructions here.
2. Open a command prompt in your new project directory (your working folder that contains the source files NOT the source code repository that contains the data store)
3. Install the LTS version of Node (I'm using 10.14.0 at the time of this writing)
   
4. Now run the 'nvm use 10.14.0' command.
   
5. Voila ... Your project folder is now using Node 10.14.0
6. You can double check by running 'node -v' command.
   

Code Editors

Your choice for a code editor is about as personal as it gets. This project contains JavaScript, HTML, CSS, LESS and JSON. Most code editors can handle these or have plugins to help. I highly recommend Visual Studio Code. It is free and has a great open source system of plugins to help you in virtually any language. It also automatically picks up your GIT settings and helps you manage your source code. Another interesting little tidbit - It is written in Javascript and hosted in Electron ...

The project contains a Cordova wrapper so that you can build and deploy to mobile devices. You don't have to do that but it is one of the features of my project so I'm sure you are eager to try it.

You will need either Android Studio (which I recommend) or at a minimum the JDK8 to be able to build for Android devices. This can be installed on both Macs and Windows but I have found some issues trying to build and deploy Android solutions from a MAC so I recommend a Windows machine.

You will need a MAC and XCODE to be able to build for IOS devices. You will also need an Apple Developer Account to sign your apps.

All of the above mentioned software is free with the exception of the Apple Developer Account which costs about $100 USD annually.

 Data Base

In keeping with the free theme for this project I decided to use a NOSQL db for this project. I chose MONGODB because of it's small footprint, easy install, and ease of use. It also stores all it's data in JSON which is perfect for a Javascript project which also uses a lot of JSON data. Again ... trying to find common ground in the project.

You can easily convert this project to use any DB of your liking, but if you want to download the project and fire it up "out of the box" so to speak you will need a mongo db available.

Ready to Go

Once you have all of the above installed and working you should be able to clone Applewood from The Smoke House Project on GitHub, follow the "Getting Started" directions in the README and start building and deploying to mobile and desktops of your choosing.

NOTE: I am in the process of upgrading the Applewood repository as I am writing this blog, so if you run into any issues or experience any problems please feel free to reach out to me anytime.

NEXT STEPS

The rest of this blog is going to focus on how I structured the project, modifications and changes to the build scripts and code and other required steps to achieve the "one code - build anywhere" functionality.  

Please comment if you are interested in a particular detail, or would like me to blog about setting up your environment or solving a configuration issue. I won't promise, but I will try to accommodate your requests.

Source Control

Almost every open source project today is available on GitHub. GIT is an open source distributed version control system. My decision to use GIT for my source control was a logical and easy choice on my part.

I am not going to detail the installation and configuration of GIT here. There are lots of tutorials out there. I installed both GIT (command line version) and GitHub Desktop (a nice GUI interface) on both of my computers. An interesting side note is that the newest version of GitHub Desktop is also hosted in Electron. (Beginning to see a pattern here?)

What I am going to talk about is my choice to use GitHub Desktop to control my source code locally instead of pushing it to GitHub on the internet.

Again, my decision was centered around finding something that I could use on both my Windows and Mac machine that would give me a consistent user experience so I don't have to remember "how to do things differently" when switching between computers. I also wanted the ability to keep my source code locally ... I'm not saying I don't trust GitHub ... I do and I use it to host this project which I call The Smoke House Project (because of my love of smoking meats in my wood smoker). I just feel more comfortable with my source code residing on my local NAS under my roof ... it's an old school thing!

Creating Your Repository

Getting GIT to work is relatively simple once you understand how it works. It can be frustrating if you just dive in and try to force it to work ... believe me ... I know! So I am going to walk you through the process ... step by step. I am doing this exercise on my Windows machine but you can do this on a Mac as well. You enter the exact same commands in a terminal window.

1. Create a directory where you want to store your collection of local code repositories. It can be on your local computer or on a NAS. I use a NAS because I want to be able to contribute to my projects using both of my computers.
2. Map a network drive to your new folder. This is not required but it just makes life a little easier.
3. Open a command prompt in your new directory.
4. Create your first repository by entering the following command:

   

   
   

5. This creates a bare repository that is ready to use.

NOTE: This creates the GIT repository or data store that holds all your source files and tracks changes.It is NOT a directory to be used for development! We will get to that in the step next.

NOTE: You can create as many project repositories as you like by simply running this command and specifying a new repository name as the last parameter.

There are some good resources here if you want to explore the git init command in detail.

Cloning Your Repository

Now that we have our repository and an empty first project created it is time to open the GIT Desktop and clone our repository to our computer so that we can start contributing.

1. Open the GIT Desktop app and select FILE then CLONE REPOSITORY.
2. Select URL in the top selection bar.
3. Enter the path to the new project directory you just created. In my case it is Z:\GIT\test-project.git
4. Enter a path to a directory on your local computer where you want the development or working code for your project to reside. This is where you will be doing code changes and making contribution to the project.

   

   
   
5. Click the CLONE button.

NOTE: A red warning box may appear in this dialog telling you the destination directory already exists. You must enter a path to a non-existing directory because the wizard will create a new folder for the project to reside in.

NOTE: Our example project is currently empty so the resulting new working folder will only contain a hidden .git folder. If the project in the repository contains existing source code, it would all be copied into this new working folder. The hidden .git directory contains all the information required by GIT to link the working folder to the repository.

Congratulations, you now have a registered working folder so that you can contribute to your new project!

Using Your Repository

Open Visual Studio Code and then select FILE / OPEN FOLDER and select your new working folder. The code editor will open your project and automatically link it to your new repository using the hidden .git directory information. You can now safely begin contributing and editing code in your project!

Now if you want to contribute to the project on a different computer you must install GitHub Desktop on that computer, then follow the instructions to CLONE the project. This process is the same as cloning a project from GitHub over the internet.

Your new GIT repository is now working and managing version control on your project on your local network instead of the internet!

Cloning The Smoke House Project into your Private Repository

OK, now you have a private repository set up with a test-project in it. Now it is time to get The Smoke House Project from GitHub and save it as a base project in your private repository. You would do this every time you wanted to use The Smoke House Project as a base or template for a new project on your local machine.

Note: The Smoke House Project is a collection of repositories and we are interested in the "Applewood" repository here.

NOTE: You must create a new empty project in your local repository before doing this! 

1. Create a temp directory to hold the clone from GitHub.
2. Open a command prompt in your new temp directory.
3. Run command :> git clone --bare https://github.com/SmokeHouseProject/applewood.git

   

   
   
4. CD (Change Directory) into applewood.git
5. Run command :> git push --mirror Z:\GIT\test-project.git

   

   
   
6. Delete the temp directory and it's contents

We now have a local copy of The Smoke House Project saved in our new repository!

Now follow the instructions above to Clone your new repository to a working directory.

NOTE: Certain folders like the "node_modules" contain content that intentionally is not included in the source code repository. This means that when you clone a project from the repository into your working folder you must populate the folders like "node_modules" before you can use the project. This is done by opening a command prompt in the working folder and running a command to fetch and populate the folder with the required packages. For our project do the following:

1. Open command prompt in root of the project folder
2. Run command 'nvm use 10.14.0'
3. Run command 'npm install'

There are also some specific instructions in the README file of the project. Please follow those instructions before using the project! You must do this every time you clone a project form the repository to a working directory!

There are some good resources here if you want to learn more about mirroring or copying repositories.


NOTE: There are two scenarios for creating a new local repository.

1. You want to clone the existing Applewood repository from GitHub and work with the completed project. (You may want it to copy snippets from or use in it's entirety.)
• Create new empty local repository
• Clone The Smoke House Project from GitHub to a temp folder
• Push the download from temp to your new local repository
• Clone your new local repository to a working folder
2. You may want to create an empty repository so you can follow along with this tutorial.
• Create new empty local repository
• Clone it to a working project folder

Ideally you will have two new repositories. One containing the original Smoke House Project and an empty one that you can use to follow along step by step. This also means you will have two working project folders, one for each project.

Theory and Design

In order to achieve the portability I needed to be able to deploy across multiple platforms I required a simple yet powerful language to host the application. Javascript was the perfect solution. Thanks to the Internet and the evolution of social media, Javascript has evolved into a development platform in it's own right.

I also wanted to keep the technology stack as close to Open Source as I could, and utilize free tooling, libraries, drivers, and all other associated bits. This included the development environment as well. I was getting tired of paying annual fees for the ability to use tools that often were a one off and then sat in my toolbox collecting dust.

So lets get started ...

Javascript is an Interpreted language so it needs a run time compiler that transforms the Javascript into machine language. I wanted to be able to develop on both my Windows machine and my Mac, so I needed a cross platform compiler.

Node was an obvious choice to host the application during the development cycle. The Node Version Manager and Node Package Manager are great tools for managing the development environment.

I also wanted a cross platform code editor so I could work in a familiar environment on either machine. Visual Studio Code was my choice. After a little research I discovered that it was written in Javascript and hosted in Electron. Perfect, my code editor was built on the same platform as I was intending to use for my efforts. This gave me a degree of confidence that I had chosen the right platform.

Visual Studio Code plays nicely with GIT, so GIT was my obvious choice for Source Control.

In order to run my projects on mobile devices I had two choices. Native development or Hybrid (HTML) development. I had worked on several hybrid mobile apps in the past and from a cross-platform point of view, it was the only choice. Cordova provides a nice clean modular approach to providing the interface between mobile hardware and Javascript as well as build scripts for each environment. There are numerous similar platforms available. I chose Cordova because I had worked with it in the past and it has a great track record.

Electron provides the same cross-platform functionality for operating system or desktop deployments. It provides a bridge from Javascript to desktop as well as the necessary build scripts.

The last piece of the puzzle. I needed a Javascript framework that provides all the basic functionality and tools to build a world class application. There are lots to chose from and I did a lot of comparison shopping before I settled on Aurelia. The Technical Benefits page on the Aurelia website gives a great overview of why I chose Aurelia. I really don't want to get in a debate over the merits of one platform vs another. I have worked in many platforms and my personal preference is Aurelia. Let's just leave it at that. I'm sure you could use any platform of your liking, but this blog will be focusing on Aurelia.

Next Steps ... Installing Software

Background

Over the years I identified a need for a simple framework that I could use as a starting point or base for multiple projects that could be deployed as a website, mobile app, or a desktop application, and is operating system neutral. In other words, I wanted to write a single code base that I could deploy to Windows desktop, Mac desktop, IOS (Apple) device, Android device, or simply run it as a stand alone website.

I tried building my own (with very limited success) and after investing a ton of effort realized the monumental scope of what I was trying to achieve. Then I discovered Cordova (Phonegap) ... ahhh ... some light at the end of the tunnel.

After lots of trial and effort and countless hours of searching and reading I finally settled in on a technology stack that could work:

• Aurelia - Pure Javascript MVC framework in a SPA format
• Node - Javascript host
• Cordova - Javascript Mobile Interface and builder for IOS and Android
• Electron - Javascript Desktop interface and builder for Windows, Mac and Linux  

Now all I need to do is put all the plumbing in place to make it work!

The Smoke House Project

The purpose of this blog is to take you along on my journey to put all of this technology together in a base package that can be reused as a starting point for other ongoing development efforts.

About a year ago I created an open source repository on GitHub called The Smoke House Project
I am writing this blog postmortem because I realized I needed to update and refresh the core code and thought other developers with similar interests may want to follow along.

I have been using this base for several projects for a little over a year and so far it has been working very well. My last project was a very large scale desktop application that required some additional code to be added to the base. As well, there have been some recent improvements in Aurelia so I thought now is a good time to update the project.

Please feel free to comment or ask questions. I am going to try to highlight as many challenges as I can but if you are stuck on something I will try to include it in my efforts here.

As always, I am open to others contributing!

You can reach me directly at my email if you don't feel comfortable asking questions here.  

push-files.js

import gulp from 'gulp';
import path from 'path';
import minimatch from 'minimatch';
import project from '../aurelia.json';
import del from 'del';
import merge2 from 'merge2';

export default function pushFiles(done) {
  if (typeof project.build.pushFiles !== 'object') {
    done();
    return;
  }
  if (typeof project.build.pushFiles.sources !== 'object') {
    done();
    return;
  }
  if (!Array.isArray(project.build.pushFiles.targets)) {
    done();
    return;
  }

  const instructions = getNormalizedInstruction();
  const sources = Object.keys(instructions);
  const targets = project.build.pushFiles.targets;

  clean(targets, sources).then(() => {
    return merge2(
      targets.map((target) => {
        return sendFiles(target, sources, instructions)
      })
  )})
  done();
}

function clean(targets, sources) {
  let jobs = [];
  targets.forEach((target) => {
    sources.forEach((source) => {
      jobs.push(path.posix.normalize(target + '/' + source));
    });
  });
  return del(jobs)
}

function sendFiles(target, sources, instructions) {
  return gulp.src(sources)
  .pipe(gulp.dest(x => {
    const filePath = prepareFilePath(x.path);
    const key = sources.find(f => minimatch(filePath, f));
    return path.posix.normalize(target + '/' + instructions[key]);
  }));
}

function getNormalizedInstruction() {
  const files = project.build.pushFiles.sources;
  let normalizedInstruction = {};
  for (let key in files) {
    normalizedInstruction[path.posix.normalize(key)] = files[key];
  }
  return normalizedInstruction;
}

function prepareFilePath(filePath) {
  let preparedPath = filePath.replace(process.cwd(), '').substring(1);
  //if we are running on windows we have to fix the path
  if (/^win/.test(process.platform)) {
    preparedPath = preparedPath.replace(/\\/g, '/');
  }
  return preparedPath;
}

main.js

//electron bootstrapper
//
const {app, BrowserWindow} = require('electron')

// Keep a global reference of the window object, if you don't, the window will
// be closed automatically when the JavaScript object is garbage collected.
let win

function createWindow () {
  // Create the browser window.
  win = new BrowserWindow({
      width: 1024,
      height: 768,
      webPreferences: {
          nodeIntegration: true
      }
    })

  // and load the index.html of the app.
  win.loadURL(`file://${__dirname}/index.html`)

  // Open the DevTools.
  win.webContents.openDevTools()

  // Emitted when the window is closed.
  win.on('closed', () => {
    // Dereference the window object, usually you would store windows
    // in an array if your app supports multi windows, this is the time
    // when you should delete the corresponding element.
    win = null
  })
}

// This method will be called when Electron has finished
// initialization and is ready to create browser windows.
// Some APIs can only be used after this event occurs.
app.on('ready', createWindow)

// Quit when all windows are closed.
app.on('window-all-closed', () => {
  // On macOS it is common for applications and their menu bar
  // to stay active until the user quits explicitly with Cmd + Q
  if (process.platform !== 'darwin') {
    app.quit()
  }
})

app.on('activate', () => {
  // On macOS it's common to re-create a window in the app when the
  // dock icon is clicked and there are no other windows open.
  if (win === null) {
    createWindow()
  }
})

// In this file you can include the rest of your app's specific main process
// code. You can also put them in separate files and require them here.

copy-files.js

import gulp from 'gulp';
import path from 'path';
import changedInPlace from 'gulp-changed-in-place';
import project from '../aurelia.json';
import merge2 from 'merge2';

export default function copyFiles(done) {
  if (typeof project.build.copyFiles !== 'object') {
    done();
    return;
  }

  const instructions = getNormalizedInstructions();
  const files = Object.keys(instructions);

  return merge2(files.map(file => {
    let targets = instructions[file];
    if (!Array.isArray(targets)) { targets = [targets]}

    let pipeline = gulp.src(file)
      .pipe(changedInPlace({ firstPass: true }))
    targets.forEach(target => {
      pipeline = pipeline.pipe(gulp.dest(target));
    })

    return pipeline;
  }))
}

function getNormalizedInstructions() {
  const files = project.build.copyFiles;
  let normalizedInstructions = {};

  for (let key in files) {
    normalizedInstructions[path.posix.normalize(key)] = files[key];
  }

  return normalizedInstructions;
}