Like most developers in the IoT space, I write code that is typically destined for Linux run-times; but I spend my days typing away on a Surface Book running Windows. In days past, when I needed to do some hacking, I’d fire up a Raspberry PI or jump over onto one of a few Ubuntu VMs running HyperV sucking up disk space, processor time and RAM. Thankfully, technology has progressed and these days, with Docker being all the rage, I just spin up a container and get to work.

I was in a very interesting workshop this past weekend on Webtask.io presented by Auth0’s Glenn Block. Webtask.io brings to the table a thoughtfully designed in-browser editor, a powerful CLI and impressive startup times that live up to their messaging of “run(ning) code in 30 seconds”. While the tool chain appears to favor reactive implementations akin to AWS Lambda or Azure Functions, they do offer the capability to run jobs on a cron like schedule.
What better way to explore this feature set than to implement a trivial IoT device simulator … so let’s get started.

Version 2 of the Azure CLI was released recently and with the added power of PowerShell, we can accomplish some truly amazing things!

In the wake of CES, This week in IoT has some interesting highlights. We saw the public announcement of Microsoft’s Connected Car initiative, built from an initial partnership with a Renault-Nissan. The Kissenger iPhone device allows you to “kiss” strangers over the internet. And though not expressly IoT, it turns out that Ultrasound tracking can be used to bypass anonymity systems. So here’s this week’s roll:

In The News:

• States Wire Up Roads as Cars Get Smarter
• Microsoft launches a new cloud platform for connected cars

In my previous post on the Azure IoT Gateway SDK, we put together a Powershell quick start script for Modbus Gateway projects.
In this post we’ll continue the exploration by connecting a Modbus compatible Beckhoff BK9100 to Azure IoT Hub, shape our data with Azure Stream Analytics and then visualizing the current position of a Dynapar optical encoder with Power Bi.

I’ve been working with the Azure IoT Gateway SDK quite a bit lately and the project setup experience is not only repetitive, but also somewhat error prone. Make a dozen settings changes here. Forget a setting there. Edit a few CmakeLists files.\Watch the build explode. Fix. Build AGAIN. Rinse and repeat.

Beating the drum of strongly typed function programming in the land of IoT is the textbook definition of counterculture. Embedded systems have been written in “high-level” languages like C/C++ forever. New players to the IoT market yearn for broad-based adoption and think the only way to drive developer adoption is to JavaScript All The Things! While precedent and low barrier to entry are certainly compelling, neither are helping us build better, more robust, provably secure or correct systems. As such, I think there is a very strong case for languages like F#, especially when leveraging opensource cross-platform run-times and sdks like .Net Core.

I’ve had the opportunity to start exploring the Azure IoT product suite lately and over all first impressions are very positive. I’ve rolled a few F# solutions for device management and simulation with good success. The next step of this project has lead me to the Azure IoT Gateway SDK in an attempt to implement a custom filtering/batching/grouping/compression gateway for a field array. Unfortunately, after adding the C++ options to Visual Studio, installing CMake a few times and adding it to my path, I was stuck. Running the included build.cmd or CMake by hand lead to a string of errors like:

I’ve been working lately on our company’s first production F# application, a simple logging application that spans between our transaction processing system and our Hadoop Cluster. Initially we’ll be logging some simple transaction meta data for internal reporting applications, but hope to expand it to other types of logging/audit data, eventually. Given that application could be reactive in nature, I thought it would be fun to play with both the MailboxProcessor and the .Net Reactive Extensions library in this implementation.

In a bit of a departure from tech, here’s some instructions on making a slick sensory swing for your sensory starved little one.

I’ve long been a fan of the deep integration of JUnit into the Jenkins CI ecosystem. Unfortunately, because most of what we build with our CI server are .Net applications, we are very dependent on a number of plugins to shim between our .Net world and the more Java orientation of Jenkins. One such plugin, that I’ve grown quite fond of, is the NUnit Plugin that does transforms between NUnit’s test result output and the JUnit format that is used for the built in Test Result Trend Graphs.

Since no one asked … A recipe for Pizza Dough!

My talk from Jan ‘16 Papers We Love(LA) is up on the YouTubes. Sadly, the abstract was a lie and my talk was about as meandering as Brooks’ original paper. Anyhow, here it is … would love some feed back!

In Brook’s ‘No Silver Bullet’ paper he argues, impotently, that “much complexity [in software] comes from conformation to other interfaces”. The whole argument, which can be found in the “Conformity” section reduces to:

One of the sweetest features of UpGuard is the ability to add custom queries for node scans. Though the standard battery of queries that the product provides is very through and well selected, it does have some blind spots.
Previously, I’ve written about using UpGuard to monitor Disk Space on server drives and to track changes to Active Directory. In this post we are going to look at a more nefarious problem that we ran into lately and leverage UpGuard to help monitor for changes.

I was recently converting one of my open source projects from Nuget to Paket and encountered a classic headache for Windows developers … adding a “.foo” directory to a project folder. The leading period is a historical *nix technique employed to prevent the ‘ls’ command from showing the directory when not using the -a flag. E.G. a folder structured like:

Some thirty years after Turing Award winner Fred Brooks’ seminal work ‘No Silver Bullet’ was published, the industry has paved over his enthusiastic skepticism, with an air of pessimism and apathy.
After tearing down the failed software development methodologies of our predecessors, I will leverage Brooks’ work to frame a critique of current industry themes such as DevOps, Microservices, and Cloud Computing. Lest however, you be consumed by an unshakable hopelessness that, as an industry, we are bound to Santayana’s Sisyphean prophesy (‘those who don’t learn from history are bound to repeat it’), I will show that Brooks’ theories are alive, well and in fact thriving in niche communities paving our way forward.

PART I

Scene notesIt’s Sunday night. 9:30 pm to be exact. The kids are finally asleep. The hiss of the record needle stings Bill from across the room as a Jawbone wafts Mancini. One rock gently spins, melting in a finger of Scotch. It’s clearly deployment time. Bill wonders where his RSA key is as he rifles through his tattered Ted Baker bag (clearly a prized possession).
He sighs heavily and begins muttering “The VPN is acting up AGAIN. Go figure. pause I’m in. pause Where’s IT? Late again …” The chat window opens.

The feature of UpGuard I enjoy the most is the ability to write and run custom Powershell queries to monitor my Windows nodes. Recently one of our system administrators came to me and asked about using UpGuard to monitor for changes on our active directory nodes. Never backing down form a chance to use some Powershell to Automate a process I figured I would give the project a go!

There is little that I can add to the conversation that has not been written before with far better prose. In fact, my assessment of the situation and the events that lead up to this morning is, at best, naive and more likely, out right rejection of the plain truth.

One of the few joys I have at work right now is that IT has given me carte blanche with respect to managing our engineering environment - they keep the power on and the network functioning.
One of the few things I detest is that IT has given me carte blanche with respect to managing our engineering environment - the only thing they do is keep the power on and the network functioning. This un/fortunate position I find myself in means keeping the infrastructure for 8 devs and QA running smoothly, which is frankly no small feat.

When modifying legacy code, one of our bigger challenges can be getting tests and test harness worked into the existing structures. As I noted in my last post on Timing Method Execution in C#, I’ve been spending quite a bit of time in our repository layer, and part of that work has been adding unit tests to old sections of the code base in preparation for some upcoming refactoring. Obviously, or maybe not so obviously, there is an art to adding unit tests to code that was not originally designed to be tested, and the older portions of our repository layer are no exception.

An awesome tweet from Jessica Kerr rolled by this morning in quite a timely fashion.

Title: The Great Decoupling:

Rewiring Brownfield Enterprise WebApps with Microservices

The are few things more awesome then that sense of hacking some personalization into a product. Whether it was huge gaudy epaulettes on my schweet Night Elf Druid or flames in the vacuform chain guards for my automation winches, I love adding that geeky special touch. So when I encountered one of the more underrated features of UpGuard and found that you can personalize it, my inner (and outer) geek sprung into action. “What feature?” you ask. Why the Timeline!

As we move our UpGuard installation into production I have a few security issues to contend with. The most significant concern is preventing a UpGuard scan from pulling down unencrypted configuration data during a deployment. In theory if a scan were to be triggered while secure data was being migrated from an old web.config to a new config, the scan could pick up IT eyes only data making it available to non-IT analysts with accounts on the appliance.
Currently the Scriptrock API does not support a “disable scans” toggle for a node, so lets go hacking!

In my last post, we looked at how to get our list of nodes from our Scriptrock appliance. Now let’s take our collection of nodes, and using Powershell’s list filtering and selecting capabilities, we can put that data to good work. Here is the original node select script:

I’m completely perplexed by people who dislike Powershell. As someone who has built 1,000+ line Powershell deployment scripts, the scripting language can be a bit like bacon flavored chocolate; you’ll try it, some folks swear by it, but for the most part my conclusion is … “meh”. That said, Powershell as a SHELL is outstanding. Frankly it smokes any experience I’ve ever had on the Unix/Linux side of the fence and it’s got everything to do with direct access to .Net. So let’s put the scripting language aside and look at some things you can do with the shell and my new favorite tool UpGuard.

As is not uncommon, I went on a pretty hard Twitter rant the other night. Aimed at the industry in general, I hit squarely on a few of the trappings of current software/technology business practices. The problem was that my approach was overwhelmingly negative, and while negativity serves to provoke and stir up controversy, it more often exacerbates the problems at hand as the targets of criticism simply entrench themselves deeper. Additionally, I was missing heaps of context that, in hindsight, makes my point complete. So, the series of blog posts that follow are simply a flushing out of my prior rant. For the sake of accountability I will start each post with my original thought, tear it down and then reshape it into the proper thesis it should have been.

Abstract:

Description:

DevOps doesn’t have to be a decree handed down from on high; in fact the best initiatives are those that start as grassroots efforts, gain the attention of management and then blossom into self-sustaining cultures. In this talk we’ll explore developer lead DevOps initiatives and how to make them succeed.

As is not uncommon, I went on a pretty hard Twitter rant the other night. Aimed at the industry in general, I hit squarely on a few of the trappings of current software/technology business practices. The problem was that my approach was overwhelmingly negative, and while negativity serves to provoke and stir up controversy, it more often exacerbates the problems at hand as the targets of criticism simply entrench themselves deeper. Additionally, I was missing heaps of context that, in hindsight, makes my point complete. So, the series of blog posts that follow are simply a flushing out of my prior rant. For the sake of accountability I will start each post with my original thought, tear it down and then reshape it into the proper thesis it should have been.

As is not uncommon, I went on a pretty hard Twitter rant the other night. Aimed at the industry in general, I hit squarely on a few of the trappings of current software/technology business practices. The problem was that my approach was overwhelmingly negative, and while negativity serves to provoke and stir up controversy, it more often exacerbates the problems at hand as the targets of criticism simply entrench themselves deeper. Additionally, I was missing heaps of context that, in hindsight, makes my point complete. So, the series of blog posts that follow are simply a flushing out of my prior rant. For the sake of accountability I will start each post with my original thought, tear it down and then reshape it into the proper thesis it should have been.

As is not uncommon, I went on a pretty hard Twitter rant the other night.
Aimed at the industry in general, I hit squarely on a few of the trappings of current software/technology business practices. The problem was that my approach was overwhelmingly negative, and while negativity serves to provoke and stir up controversy, it more often exacerbates the problems at hand as the targets of criticism simply entrench themselves deeper. Additionally, I was missing heaps of context that, in hindsight, makes my point complete. So, the series of blog posts that follow are simply a flushing out of my prior rant.
For the sake of accountability I will start each post with my original thought, tear it down and then reshape it into the proper thesis it should have been.

Somehow the impermanence of life in the theater made it OK to talk about “the future”. Most of us talked freely about what we were going to do when we finally left. Start our own businesses. Go to work for that ‘other’ shop. Go back to school. Get out of Theater all together. We all changed jobs and frequently; some of us every 6-8 weeks and the impermanence was, comforting.

As I noted in my last post, I have been struggling to implement a LINQ provider over a Solr index. While the adventure has been trying at times, for the most part been, it has been quite successful. My final hurdle, one that I had been struggling with for the last week or so, was quite confounding … when the $skip option was used, it was seemingly double applied. To fully understand this behavior, let’s look at a few examples:

1. A query string of ~?$top=10&$skip=10 over a set of 100 results yields 0 results.
2. A query string of ~?$top=10&$skip=8 over a set of 100 results yields 2 results.

Over the last week I have been working on implementing a custom Linq provider that wraps a Solr search query and returns a composite projection of the solr search results and some underlying Hbase data. Just to add to the madness, the composite projection is a child element of a WebAPI endpoint and is made query-able via Odata.

Fresh off the plane from a few days at Strata + Hadoop World, I thought I’d draft a quick wrap up of the conference trade show. This being my first attendance, expectations were quite high and I was unfortunately left frustrated and a bit disappointed.

Recently, I have been preparing a whole series of blog posts on how developers can lead DevOps cultural changes within their organizations. In researching for these pieces I have come to the realization that there is an unfair but immediate assumption that discussions of cultural change are a response to an existing dysfunction or cultural defect. While there is obviously a just cause for that assumption, and I don’t mean to be trite when I say this, I think there is a great opportunity here to think and act differently.

While industry leaders espouse the benefits of Continuous*, agility and a DevOps culture, many an enterprise developer stands in a fallow field searching for greener pastures. The popular perception that transformation must originate as a corporate decree or be sold through consulting hours is flatly wrong. Cultural changes can be home grown, evolving from the initiative of a passionate individual with just a little planning and ingenuity.

Building a team, one that is high functioning, efficient and effective, is without a doubt the hardest part of being a manager. Touching on every soft skill imaginable, the tides of team building, the hiring, the firing, the nurturing and the nagging, will give one the opportunity to equally celebrate and loathe the responsibilities. Finding balance through the swell is a matter of accepting that organizational evolution is natural, desired and can be leveraged to ease the challenges of team building.

After listening to the most recent DotNetRocks roundtable and dozens of other talks lately on DevOps, there is a presumption that the cultural change is a forgone conclusion, a decision already made by management, emphasized by statements like: “You don’t want it to break in production, give the developer the pager”. While this version of the story is a great line for consultants to wield on management, don’t forget about developers who see DevOps as on opportunity to enhance the value they bring to a business.

I just finished the DotNetRocks Episode:1080 with Bryan Hunter where he espoused the benefits of the Elixir language. After an hour of ear drum caressing, I sat down to ‘mix’ up some Elixir. Of course, having little familiarity with Sublime Text 3 (what appears to be the defacto editor for Elixir on Windows?), I was stumped before I could ever lay a line outside the REPL.

I wish I had written a summary off the back of last year’s “I Love APIs” conference. Of what I remember, it was primarily about what APIs are, the great possibilities for APIs, what they can do for your organization and so on. Nominally, introductory material.

Since I have been digging theory pretty hard lately I thought I would drop a quick blog post with some of the research, lectures and such that I have been working through. I highly suggest perusing this stuff …

A few weeks ago I had the opportunity to execute a small business rules engine for a client feature request that was done as an internal Domain Specific Language (DSL). The novel aspect of the project for me was not the harnessing of extension methods; that, while not elementary, was trivial compared to the application of functional design/programing, specifically compositional concepts. The resulting product was a terse, highly readable module that demonstrated the ability to fold functional approaches into an object oriented language and architecture. Most importantly however, this project marked my first real “Ah Ha” moment with functional programing & design approaches.

Dear Diary,

Thoughts on organizational communication …

I have been reflecting over the last few days, what it means to be a “lazy” developer, inclusive of both positive and negative connotations.

The day (and lately, night) job has been focused on a data api backing a new business insights tool. Despite concerted efforts by nefarious interests, we have managed to keep the API resource centric, only leaking behavior where absolutely necessary. We are starting the process of putting internal customers in front of the tool, and as such, I have become increasingly aware of the performance issues, of which there are quite a few.

I have a fun little project going on right now that involves taking a portion of a web page, loading it into PhantomJS, passing a lump of data to it, taking a picture of the page, and displaying that image in a RDLC report. There are like a ba’gillion different ways to accomplish this, so the below is a hacker’s proof of concept.

If you could pick one word, and only one word to describe yourself, what would it be? What word would you own? What word wakes you up and puts you to bed? What word drives your decisions, challenges your peers and inspires your team? Go ahead and think about it for a minute, I can wait.

To solidly rip off DotNetRocks … today’s installment of “Better Know a Framework’s Methods”, we bring you the Generate Password method of the Membership class in System.Web.Security namespace.

Continuing the epic series on Hbase, Thrift and C#, this installment is the code for a batch scanner by leveraging inheritance and generics to provide a broad based solution. If you have not done so please check out the other parts of the series:

Continuing the epic series on Hbase, Thrift and C#, is this installment where we will review building our first row scanner by leveraging inheritance and generics to provide a broad based solution. If you have not done so please check out the other parts of the series:

In my last post, HBase, Thrift & C# - First Connections, I mentioned and included in the code, references to a Session Pool Manager. I wanted to take the opportunity to discuss some of the behavioral aspects of this class. Unfortunately, because this work is completely unrelated to any open source projects, I don’t feel comfortable sharing the actual code. That said, I won’t hesitate to talk about how it works and discuss some of it’s features, in the event you are looking to implement a session manager. Don’t forget to check the other posts in this series.

I seemingly have a bad habit of never writing part two of a series. So to buck the trend, this piece is a follow up to NuGet Servers, HBase, Thrift Code Generation and one sweet Jenkins CI Build. You don’t need to read the whole post; but, familiarizing yourself with section III and on is important. Don’t forget to check other posts in this series.

Its always a bit of a joy when pet and client projects align on similar goals, the proverbial ‘killing of two birds with one stone’. Admittedly though, I applied a touch of coercion to get these in sync.

Every few months a new article on interviewing or on being the interviewee goes viral, finding its 10 minutes of fame; I hope this one is no exception.
Interview articles often focus on what kinds of questions will be asked or inside lines on how to answer behavioral questions, etc. etc. For the most part, all of it is crap. Most interviews are crap. Most managers and team members don’t know how to interview for their teams and most prospective employees don’t understand their part in the encounter. Hell, these days getting to the interview at a large corporation is less about who you are and more about your resume’s SEO. What follows is the first in a series about interviewing, being interviewed, and team building/development.

You are in your mid thirties, wife, kid, home, great job, everything is humming right along, and yet … you are stymied in a micro existential crisis.

Last night’s #ArchitectureMadeEasy tweet included a wonderful little video from the This Old House series.

I just finished a block of code that, while not ground breaking, is well worth recording. The problem setup is:

For better or worse, the SCM I work with is Mercurial provided through Fogbugz’ Kiln. Yeah I know it’s not GitHub, but I was ignorant of that decision as it was being made so, yeah. Reviews, complaints, flamewars aside, working with it has not been bad and it has a reasonable feature set. Of note is the ability to program web hooks to fire on a push to a repository.

My Jenkins installation had turned into a bit of a saga lately. I have been interested in trying out Artifactory by JFrog to store not only our produced binaries; but, additionally to proxy external dependancies and provide a host for general software used across the enterprise.

In this post I want to talk about template builds, stable builds and branch/feature builds for Jenkins.

I have been doing some serious data digging over this last week - mostly related to tracing some arcane functionality and an associated issue with a data feed. The work was pretty time consuming, so while the SQL was running, I had some time to think and experiment. Though we knew there was an issue with the data, its impact was not really visible through our reporting tool.
Low number of occurrences. Steadily climbing counts, but with pretty low deviation. I needed a way to expose what we thought was happening via a meaningful and actionable indicator.

I am deep in the throws of a slow migration that is laying the groundwork for a continuous integration style build => unit test => deploy => integration test, pipeline. I have shaped Jenkins enough to get builds and unit tests to function smoothly; however, recompilation is still happening during the packaging step using an existing NAnt build system and not Jenkins.