Ethereum DevOps with Truffle and Visual Studio Team Services

Hi, I’m David Burela. And today, I want
to take you through how I’ve set up a DevOps pipeline for Ethereum development
by using a combination of Truffle and Visual Studio Team Services. Now, I did do
a blog post on how to do this about a year ago, but it was very convoluted and
clunky and that was just because of the limitations of the technology at the time.
Now, there have been a number of improvements since I last did it. Truffle
is much easier and simpler to install now, NPM has a cool new command, and Visual
Studio Team Services now supports hosted Linux build agents, which means I don’t
have to set up a virtual machine anymore, I can just use their throwaway ones. Now,
all of the steps I have here should work on any build system such as Jenkins or
Team City, but what I’m going to show you today is how to do this on Visual Studio
Team Services because that’s what I’m most familiar with. Now to do this, it happens in three main
steps. The first one is configuring NPM to go install on some packages that we
need. The second one is to configure our truffle.js to get it ready for the build
agent, for a remote server to build this for us. And then finally, we’re going to
set up our build server by setting up the tasks that we want it to do. There’s
only four or five simple tasks there. Now, if you get lost or you just want to
have something for your build agent to point at so that you can just test this
out yourself, I’ve actually created a sample report up on Github. So if you
clone that and then get your build agent to point to your clone then you
should be able to just get started. Now, let’s start at the beginning. I’m
going to create a new Truffle project, so DevOps Demo, empty folder, Truffle
in it. And this is just going to create a simple Truffle project for us. If I go
Code., that’s going to launch Visual Studio code for us with the context of
that folder and we can see its standard. We’ve got our contracts and we’ve
got this truffle.js. So let’s start at the beginning. It says “Check if you
have a package .JSON file.” Now, we don’t have one, so we need to
create one and we can do that with NPM init. And WordPress is trying to be
helpful so that’s getting in the away. Okay. “NPM init.” Great. And if we have a
look that could create just a standard template. Now, let’s run that next command
and if WordPress would stop being useful. Stop at WordPress. Copy. Let’s
actually test that NPM install. Yep, it’s all there. WordPress. Now, if we
let that go, what that is actually doing is going and installing as a Dev
Dependency all of these tools, then our build server can go and use these tools. Now, the problem that I had before is
we’d have to install these things on globally because these would
take a long, long time to install, and so I’d have to set up a virtual
machine with these pre-installed. Now, because the Truffle have gone and
made these things a lot simpler to install and takes a lot less time, we can just go
install these as Dev Dependencies, then I add build server when
it goes and does this restore. Now we’ve got these tools that we can run within this project, and we don’t
have to have them installed globally anymore. And so that makes that much,
much simpler. So we’ll let that go. And so what we’re
installing is Truffle, so we can do that build and compile and the testing to do test RPC so that we can have
a little in-memory block chain that we can test against mocker to run our unit tests,
and then this J-Unit reporter so that we can spit out the XML file of our test
results. So we need those four things, and that’s going to take a few more
seconds. We’ll just let that finish. And while that’s finishing, let’s have a
look at what we would want to do next. Now, the next step is we have to update
our truffle.js so that we can do two things. Well, there we go. Well let’s have a look at that first.
We go and look at their packages .JSON, now we see our Dev
Dependencies, we’ve got those four things there. So that’s great. Now, if we go back
to here, I can get back to this truffle.js file. We’re doing two main things. One,
we’re adding in a new network definition so that we can tell Truffle to go and test
against this network or do migration against this network. And the cool little
hack here is we’re going test our RPC provider, which means that Truffle will go
and spin up a new test RPC instance, just for the lifetime of that command,
so Truffle Test or a Truffle Migrate. And once that command finishes it
would go and just throw it away again. So that’s great for our build server
because we don’t need to install test RPC globally anymore, we don’t need to
spin it up before we run our tests, we don’t have to worry about tearing it
down after our test finish because this would just happen automatically and any
lift for the life of the context of that one command. And the second thing is we’re just going
to put in some definition at full mocker. So this is getting it ready for our build
agent. And we’re going to leave it as the spec reporter so that when the developer
runs it locally we can still see the results. But we’re going to go and just
predefine what the output file will be, so it’s ready for our build server. Now
let’s just copy these bits one at a time, truffle.js. We’ll go and put that test RPC
definition up the top where we’ll add in a new network. Copy a bit too much. Okay. So we’ve got
that new network, and so after networks we’re going to add
this section for mocker. Great. That all looks good. Now, let’s
make sure that I haven’t broken anything. So if we go “Truffle test – – Network” and
use the name of that network that I had here, local test, hit “Run.” Now that will
go…have Truffle compile all of our contracts, then it’s going to try to
connect to our local Test Network, which here means it will go and spin up our test
RPC in Memory Network temporarily, and then it will go and run all the tests.
So in a second we should see the test start executing. There we go. And all the
tests. And because we’ve still left the spec reporter here they go and output here
so our developer can keep on running our tests here locally. Now, let’s move on to… So we’ve got
this ready. Let’s go and have our build server build this. Now if I go and create CC source demo. We’ll
Call it “DevOps demo.” “DevOps Demo,” and let’s set that
default for “No” for the “Get Ignore,” have a look at our changes. Yep,
that’s all the files that we want. Have a look at our truffle.js. Yep,
that’s what we want. Package.json, there’s our Dev Dependency. Okay, it all looks good. So we’ll
call this the “Initial Check-in,” commit it to master and
we just publish that up to Github. Publish. Okay. Now, I’ve gone and pushed
my code up to Github ready for us to do it. Now for the building of it, now
Microsoft, there’s a cool thing called Visual Studio Team Services, and you
can go and sign-up for a free account for this. And if you sign up for a free
account, you can go and host your code up on here if you want. I’m not doing that
today because as you can see I’m using Github to post a code. But what they
do let you do with free accounts, you get 240 minutes of build time for
free. And so we’re going to be using this to build our code. And these builds only
take one to two minutes and so that’s for 240 minutes for free we get
a whole bunch of builds out of it. Now, once you go and sign up for a free
account you’ll be in your projects folder, so if I create a new project. Let’s
call this “DevOps Demo.” Now, we can get it to host the source code for
us, but I’m using Github so it doesn’t matter what we choose there. I only
want to share with team members. That’ll take a couple
of seconds to create. Move that there. And
let’s move this over here so that I can look at these things
as I’m going through. Okay. Now, it’s set that up. We’re going
to go to “Build and Release,” go to “Build,” and it’s going to tell us
that we don’t have a build definition yet. So we’re going to create a new definition.
And there are a whole bunch of defaults here, so if you’re building IOS
applications or Android applications or web apps or Node.js projects, but we’re
just going to create a new empty process. Now, it wants us to set it up. This one is
important. It asks us where…what type of build agent do we want to build this.
Now, I’m going to use a hosted Linux preview, this means Microsoft hosts
this for us, that just go and spin up a Linux build agent for us, do the build
and then throw stuff away for… If we go to “Get Sources,” if I go to… So if I clicked this
project, If I was using the Gitrepository within Visual Studio
Team Services. I could use that. But because I’m using the one
on Github, we’re going to set this up. So if I authorize… There we go. And
we give it a second for it to go and pull down all of the repositories that
I have access to, which is quite a few, so it might take a couple seconds. There
we go. If I scroll down to David Burela. And what do I call this? “DevOps
Demo.” Master branch. Great. And my other cool thing, if we go
over to “Triggers” and we can turn on “Continuous Integrations” every time I go
and change my solidity smart contracts and do a push, then this is going to automatically detect
that new code is being pushed to Github, pull down the source
code for you and then start doing the build. So if we go over to Tasks,
we can now start setting them all up, and if we have a look, the first one
is to do that NPM install, so we’ll go and restore all those node modules
for us. If we have a look at it, it’s just standard NPM install. Next one
is a shell script. If we add in that shell script, change this to version
three so I can get this inline bit, and now I can just copy this. That
looks good. We can give this a name. What do I call this? There we go. Let’s
scroll down and we want to add in another shell script. Version three. So I can get that
inline again. That looks good. What do I call this? “Truffle commands.”
And the last step, which is to just simply publish those test results.
Publish those test results. Okay. Now, that’s it. Let’s have
a quick overview of what’s here. So again, that NPM install is going to just restore all of our
Dev Dependencies, so that we can execute those commands. This
environment details and config is just here as an informational thing. I like
having it so that if things break, I can see what version of node the build agent
had, the NPM version and the version of Truffle. Now once that
spat out, what we do is we do a string replace inside of that truffle.js file, so that will change it
from that spec reporter over to our mocker J-unit reporter. Yeah, that’s what
I thought. WordPress didn’t copy the entire line for me. That would have been
bad. And then after that, we go and do all those Truffle commands. Now, this MPX
command is really cool. What it does is it will first check if
you’ve…well we’ll go and run this command if you have it inside the node modules folder. Now, because we have it as a Dev Dependency,
we’ve installed it. if we go MPX Truffle it will go,
looking at the node modules folder, see the Truffle that’s there and then just
execute it. So this is how we don’t need to install things globally anymore. So
we’ll compile them, I go and test that our migration scripts are all correct and
we push them out to that test RPC, that gets blown away before we do our next
thing, which is doing an MPX Truffle test. Again, against that local test Network,
and then we’ll go run all the unit tests. Now, I could probably just run that one
command and it will do those above two, but I like being explicit just so if
something breaks I can track down where it broke. And the final step is just to search for
any test XML output files and then just publish those test results. So let’s save
and queue this and everything should be working well. Queue. Good, 94 has been
queued, and let’s go watch it while it’s doing it. Now, crossing our fingers
everything has worked well. If not, we just go back
and try to debug it a bit. Here we go, we’re running. The first
thing it should be doing is pulling down a source code from Github, it’s done that,
it’s doing an NPM install to restore all of those Node Packages, so Truffle will
test RPC etc. This will take a couple seconds for it to go
and pull down all those packages. Done. So it’s bidding out our environment variables, so we’re
in Node version eight, NPM version 5.3, Truffle 3.4. It’s now doing our Truffle
compile. Yep, it’s doing our Truffle migrate, those migrations are looking
good, and now it looks like it’s about to run our tests. Running, running, running. Cool. It’s done it. It’s publishing our
test results. Hey, that’s looking good. Now let’s go look at our tests and change
the outcome to show us all unit tests. And we can see all our unit
tests are here. And there we go. So that took 15 minutes to go and set up
a fresh brand new Truffle project, to go and get it configured, create a brand new
project in Visual Studio Team Services, and get our entire build set-up configured
and running. So I hope that found it useful. I tried to do it as a video, as
well as a blog post so that you can see someone do it step-by-step. So I hope that
works out for you and if you do end up using this for any of your projects, I’d
love if you left a comment and told me about it because I’m just interested
in what people are building. Thanks a lot.

One thought on “Ethereum DevOps with Truffle and Visual Studio Team Services”

  1. Thanks again for creating this! I was looking at getting into development and this video lays down great groundwork for developing dapps. Ironically I'm learning to get into DevOps as well

Leave a Reply

Your email address will not be published. Required fields are marked *