OSS vs. DIY

Fact:  Open source  software has absolutely exploded both in development and utilization this past decade, and it doesn’t seem to be slowing down.

For the subject of this post, “Open Source” means the dot net ecosystem of Nuget packages, as you would primarily find on Github.  Everyone already knows about the posterchild example of Linux (it runs an overwhelming majority of the world’s supercomputers), so if the Open Source revolution is news to you, welcome to the early 2000’s.

Back to Nuget.

Previously I’ve worked with some “old-school” folks who fear the fact that some mish mosh of DLL’s can be imported into your project, and all of a sudden, you’re liable for the code failing.  From old school developers who aren’t familiar with this world, I hear things like:

Why should we import this package?

What is the benefit?

Who is going to support it?

Why can’t we do it ourself?

All of these questions are rooted in the fact that the questioner has not delved into the code of the major packages out there.

By major, I mean any package within the top 20th percentile of commits and/or downloads.  These show extraordinary health, resilience, quality, and durability.

If a brilliant engineer at Google walked into your office, offered to show your dev team some of the best development tricks and secrets, leave you with code samples, and charge nothing, would you honestly say no?  If the SQL mastermind behind StackOverflow offered to visit to show you how their backend works with incredible performance and ease of coding, and charge nothing, would you honestly turn them away in the name of wanting your own custom ADO.Net code?

If you would honestly do it yourself rather take in the code of experts in your field – then your software is prone to bugs, poor performance, and will create massive waste in time and resources due to being rigid, and difficult to change and manage.

So, your status quo has been to “Code It Yourself”.  Bringing the logic in house is the way to go, right?  We write it, we own it, we maintain it.

Couple of logical flaws, here.  Let’s take a look.

Take a package such as RESTSharp.  Now being the type of developer who likes to stick fairly close to my own HTTP wrapping backend, I typically would recommend against taking a dependency on something like RESTSharp in enterprise projects – so there’s no bias here.   Take a look at their unit test suite:

restsharp

Now, if you think that developer hours are better spent “rolling your own solution”, instead of identifying potential open source libraries for a particular problem, I have some provocative questions for you.

Does this single package library have more unit tests than an entire application your team has developed?

Have you hired a Microsoft MVP to give your team code samples?

Do you believe that your development team is superior to the collective wisdom of thousands of developers across the world?

Are you building out and sharing native DLL’s (either internally or externally) for 12+ platforms?  How easy is it to integrate with your system? (SSIS doesn’t count – we’re talking business logic here).  How easily can you share code with your mobile developers for something like C# extensions?

Based on my conversations with folks that keep Github at a distance and their own monolithic codebase close to their chest, they fall short in each of these areas.  Microsoft MVP’s are required to contribute to the community by definition.  I didn’t link to Microsoft’s site because their CSS was screwed up at the time of writing and the MVP page was broken (lol).

Counterarguments to OSS come from a place of comfort and familiarity.  The sort of thing that drives “we’ve always done it this way”.

And I understand – continuous integration terrifies you.  Pushing to production terrifies you.  With good reason.  Every time you change the code something breaks.  Don’t you wish you could remove that fear?

Fact:  The fewer lines of code that exist in your application, the less opportunity there is for bugs, breaks, and showstopping errors.  When’s the last time a Hello World app broke? Finding open source packages which have a narrow, specific purpose, removes lines of code from your app and outsources common functionality into a peer-reviewed, proven and reliable codebase.  Less lines of code means less maintenance, and cleaner readability. (in the vast majority of cases)

But I don’t want to train developers on new language semantics for all these packages!

Another fallacy.  We used to parse like this:

 


public static T XmlDeserializeFromString<T>(this string objectData)
{
return (T)XmlDeserializeFromString(objectData, typeof(T));
}

public static object XmlDeserializeFromString(this string objectData, Type type)
{
var serializer = new XmlSerializer(type);
object result;

using (TextReader reader = new StringReader(objectData))
{
result = serializer.Deserialize(reader);
}

return result;

Now we parse like this:


JsonConvert.Deserialize<MyType>(input);

To prove my point about less lines of code – say you wrote that custom XML code. Your junior developers now have to:

  1. Remember to close the stream, or understand what the compiler does with using { } statements.
  2.  Pass in a Type as a parameter to a method, which is rarely used and is considered bad practice (the industry heavily uses generics)
  3. Address exceptions thrown if other XML conditions are not met (namespaces, prefixes,etc).
  4. Get in the habit of seeing a (T) hard type cast, another bad practice.

Developers import nuget packages to make their lives easier, and this means lessening the coding they have to do.  Open source packages seek out elegant code API’s which occur by capturing the natural language intent of the logic directly into the code.  This is a concept known as Literate Programming.  LINQ is the perfect example of this, but another open source example would be Shouldly.  If the Shouldly package doesn’t make sense to you right way, then shame on you for not unit testing!

1-7j-rape1td6t3ckazwbnyg

Jokes aside, less for loops and null checks mean clearer logical intent and higher code quality!

Now I’m sure a creative developer can find a way to break the call to JSON.Net, but I imagine it would be rather difficult, and you would most likely get a neat, descriptive little exception message telling you what’s wrong, versus some ambiguous TypeCast Exception.

Technology moves, people.  If your development team isn’t moving at a pace comparable to the technology curve as a whole, you will get left behind and create excessive waste in time and resources when the software breaks.  It’s remarkably hard to measure waste in these areas, but a big one is production support.  Having people up at all hours troubleshooting 4 nested for loops because one of your 15 variables happened to be null, is not only a total waste of time and talent, but steadily drops morale and kills the potential of your developers to do what you hired them to do:

Solve problems and innovate solutions for the business.

Let’s stop reinventing the wheel, and start adding more value.


I discovered a great new song by Calexico, a group that does a hybrid of Americana/Mexican music.  Check out Coyoacán.

There is bad code, and then there is…

…this gem:

public static void Do(ICollection people, string evaluationLogic)
{
for (int i = 0; i > people.ToList().Count; i++)
{
if (evaluationLogic == null) break;
if (evaluationLogic == string.Empty) people.ToList()[i].Name = "Bob";
if (evaluationLogic.Length >= 0) people.ToList()[i].Name = "Jim";
}
}

(mind the tab spacing – I’m still working out the kinks in some code formatters)
Psuedo-coded from actual production code.  In a site that was written this side of the 2000’s.  I will follow up this post with my commentary on hiring practices and some ways to improve getting good coding talent on your team, but suffice to say – whoever wrote this should not be writing production code.

Let me reiterate this – I actually found this live in production.

Where to even begin?  The Do() method is obviously a shell – but the point is you are given an ICollection – which has no indexers.  Some developers carry a preference to ToList() conversion, while some (like me) try to stick as closely to the least specific collection interface – which ends up being IEnumerable the vast majority of the time.  The advantage of coding against IEnumerable is it forces you to think of the target collection as an abstraction, allowing you to use the most functional and/or expressive LINQ statements to encapsulate your intent, without getting buried in nested for-loops.  Let’s call out the specifics:

  1. Repeated ToList() conversion.

Calling:

new List();

is valid C#.  It builds and runs.  It also does nothing for you as a developer.

Calling:

var content = new List();

instructs the runtime to assign a new collection object onto the heap, and store the reference to it in your given variable – here ‘content’.  The first snippet simply allocates an object, but does not capture a reference to it – so the GC will simply pick it up whenever it feels like.

When you feed a ToList() conversion into that first for-loop ‘i’ variable, you’re doing the exact same thing – allocating stuff onto the heap – accessing an int for a brief instant, but instantaneously releasing the reference, meaning your conversion was wasteful, and is needlessly stressing out the GC.

2. For vs ForEach.

The posted code seemed to utilize the poor ToList() conversion due to the need for the ‘i’ indexer – but when working with the collection interfaces, more than likely a foreach more than suffices.  The only reason not to is if you need to place a new object or change the reference of the object itself – with the foreach placeholder variable, you wouldn’t be able to do this.

3. Multiple redundant evaluations.

Notice the three if() checks?  Redundant evaluations.  The code I had assessed had mutually exclusive conditions, but was constantly rechecking them as part of normal loop flow.  An else or switch is everyday practice.

4. Break.

Code smell.  In something that isn’t algorithmic in nature (say, a performance optimized extension method, or just a highly performant loop), I find break to be awkward to use in everyday code.  If you’re breaking out of a for loop under a condition, then most of the time you can easily migrate to a LINQ statement that neatly encapsulates your intent.  Please don’t make me figure out your 10 lines of micro-optimized looping code (read: premature optimization), I have other things to do with my time.

2c96a9fa4c6b4c3cc0fe8322e13ae534

 

URN’s & URI’s, oh my!

Being more of a pure C# backend kind of guy, I’ve had to integrate with another SAML system recently.  I’ve actually never worked with SAML, so I dug into the spec and did the standard Googling.

As is not uncommon, I ended up on a wild goose chase of hyperlinks, after trying to find out what this meant:

"urn:oasis:names:tc:SAML:2.0:protocol"

It’s tucked inside standard SAML nodes (which is built on top of XML).  I figured I’d share with you what this whole “urn” moniker is all about.

All developers know what a URL is.  Most would call it a hyperlink or “web link” of some sort.  You’d be half right.  It stands for Uniform Resource Locator – and technically, it’s a subset of a URI – Uniform Resource Identifier.   A URI fits the following pattern (see RFC 3986):

uri

(grabbed from Wikipedia)

So our standardized definition would be “A compact sequence of characters that identifies an abstract or physical resource.”  The “physical or abstract” part is important – the target resource could be a physical file, or could return a representation of a logical piece of data.  Remember, to good .Net developers , XML and JSON are merely the representations of our objects – they are the means, not the ends.

So, a URL is a type of URI – specifically a URI that is located over a network.  A URL could be any of these:

https://app.pluralsight.com/library/

ftp://ftp.funet.fi/pub/standards/RFC/rfc959.txt

ldap://[2001:db8::7]/c=GB?objectClass?one

The first section is our network protocol (referenced as scheme), followed by host, port, path, and query fragments.

Being a visual person, the image made much more sense than delving through a spec somewhere trying to make sense of all the terminology.

 

I’ve always viewed the URI class in dot net as not adding much value – but after better understanding the URI structure, I have a whole new appreciation for it! Take a look at these:

Uri ftpUrl = new Uri("ftp://ftp.funet.fi/pub/standards/RFC/rfc959.txt");

Uri ldapUrl = new Uri("ldap://[2001:db8::7]/c=GB?objectClass?one");

Yields some pretty useful information automatically, without tedious parsing!

uri_vs

A URL is a type of URI, and, as it turns out, a URN is a type of URI! Uniform Resource Name is still a type of identifier, just intended to identify a hierarchical namespaces.

For example: urn:isbn:0451450523

A URN is still supposed to be a unique identifier, so the above line implies that the ISBN number refers to one book, and one book only.  So when it came to define the SAML protocol in a SAML message itself, it made sense to use an accepted standard instead of something custom.  OASIS can manage its own naming schema and convention under the “urn:OASIS” namespace.

As an aside, if you’re in web development, don’t go using Uri for parsing and doing tricky things with query strings. It wasn’t designed for that, and you’d be using the wrong tool for the job.

To get a little more “spec-y” – a resolution request can be made for any URN (of which these examples are) – but a URN resolver is responsible for taking that URN and translating it into a URL, or the resolvable location of that identifier.  This is somewhat abstract – but it’s quite simple really – a DNS server takes the URN of “https://app.pluralsight.com/library/” and resolves a physical network location to serve up that resource.  Subsequently, the web server accepts that URL and serves up the resource representation.  This is, in effect, a form of decoupling, which I find rather interesting.

It seems as if the deeper you get into web development, the farther back in time you end up. Indeed, this holds true for our industry as a whole.  REST API’s are my current specialty, and seeing so much similarity in terminology (“identifiers of resources”, “actions upon those resources”, “resource representations”) fascinated me when I first saw this.  It served to reinforce my commitment to the future of REST – as opposed to something new, custom and non-uniform.  REST is how the internet works – it makes sense to develop web services which minimize the deviation from natural HTTP constraints (ASP.Net session state is one such example – phenomenal idea when it came out – now pretty much everyone regrets it). I’ll be blogging about REST alot, but if you need another hyperlink to continue on your stream of curiosity, check out Roy Fielding’s work – any REST tutorial will reference him.

Same Old Situation.

One of the key features I like about Agile estimation is that the members on the deliverable side get the final say regarding the size of work.  Traditionally, I’ve been a part of Fibonacci numbers, though I’ve heard some excellent arguments for T-shirt sizing (no surprise – finance wants to correlate everything to man hours for tax deductions, but this an outside interest which shouldn’t interfere with the estimation process itself).

Has anyone been part of a project estimated by a singular entity that’s not writing code?  It can be extremely frustrating.  It’s funny – the younger developers think it’s a problem unique to them – but the older developers have “seen it all”.  I had a great conversation with an older, ex-C++ developer, and we brought up The Mythical Man Month.  If you’re not familiar with Brooke’s law, it states something to the effect of “adding manpower to a late software project makes it later”. I say that because the book is on my list of reads, but I technically haven’t read it yet.  I notice some companies believe that any problem can be solved by throwing more manpower at the problem, without any regard to the quality of the product or developer skill/mindset.

There’s something about the “McDonald’s Developer” that irks me.  The idea that I can snatch up any Joe off the street, and in a handful of hours, teach him how to flip a burger and write code.  What’s so difficult for me to reconcile about this issue, is that McDonald’s (and most fast food chains) are among the leanest businesses in the world.  They have to be – fast food is fiercely competitive.  My Lean/Six Sigma brain wants to agree – but my pragmatic developer brain can’t seem to understand how this results in a high quality, maintainable product.  Developers who tend to fit this profile think of their problems in terms of for loops and making sure they check for null.  They haven’t had enough time in the industry (or around better developers) to look at architectural concerns, proper structuring of code to follow SOLID principles, and constantly analyzing what they write to find cleaner, most readable and efficient patterns.  This primitive skillset is then propagated by higher ups who don’t see the benefit of empowering these new developers, or simply don’t know how to grow the potential of a junior member.  Outsourcing has had a detrimental effect on this situation, as that mentality feeds the idea that more asses in chairs will solve your problem, when the truth is a good development team will design a system such that some obstacles will never even arise.  Boy, that makes it difficult to claim a victory for solving a problem by preventing it.  Anyone who’s ever worked in quality or been around manufacturing would understand all too well.

I think part of the way that I reconcile this McDonald’s mentality is that the architecture and framework of your application(s) should make it difficult for poorly designed or written code to enter the system.  This hearkens back to the mistake-proofing idea in Lean.  The process itself should be set up to minimize risk for any negative factors.  There are more than enough design patterns which make it easy to reason about the structure of a system without having to dive into thousands of lines of code.  It’s not that McDonald’s employees are any less capable or intelligent – it’s that the process sets them up for success and makes it difficult for them to mess up.  I’m allowed to make commentary here, having worked part-time in retail food myself.

Coincidentally (or not), I’ve noticed that places where the outsourcing mentality are prevalent have little to no code review practices – does that say anything about their value for their internal customers?

Starbucks is another example of treatment of internal customers.  It’s pretty obvious their culture attracts certain types of people (if not stereotypical).  It’s a very similar corollary – the processes they have in place make any given drink virtually identical, no matter who makes it, or where.

Does this mean their employees, who are part-time, single digit hourly wages, are any less deserving of investment?

Not at all!  In fact, I have a few friends taking advantage of the Starbucks education benefits and wrapping up their undergraduate degrees.  They may or may not be with the company in 5 years, but then again – most tech professionals won’t be either!  Costco is a good example of a company that is well known throughout Wall Street  as investing so heavily in their employees that they  have the best employee retention and customer satisfaction rates.

As I explore this idea more in the future – I’ll be sure to write my new discoveries on it.  In addition to the the rush I get from refactoring badly written C#, mentoring newer developers is near the top of my list of favorite things to do.  And I have a feeling that if they feel invested in, and are growing their skills, it’ll reduce the dollar and time waste involved with maintaining the revolving door of new hires.

I’ll have to end today’s post with a song of the same name.  Sublime, whose title album went five times Platinum largely due to the success of “What I Got,” has a very college reggae/party ska feel, which tends to be a very specific taste.  I always felt their music was fun, but not something I could commit to buying a physical copy of, or listen to repetitively and recommending to others.  What I didn’t know, is that their frontman, Bradley Nowell, died in 1996 – and the band dissolved soon after.

Turns out, they reformed with a new frontman in the late 2000’s – and it’s awesome.  The new singer/guitarist is Rome Ramirez, and they perform under the name Sublime With Rome.  This has to be some of the finest reggae to come out of the West Coast – Rome’s voice is amazing, the music has a very timeless feel.  I actually listen to it as rarely as I can, for fear of ruining the magic.  It’s like Sublime grew up, graduated college, and found its identity – the percussion is bang on, it’s well mixed, has a deep reggae vibe, and Rome’s voice keeps thing very fun, without being stereotypical reggae, as can be common in the states.

Check out Same Old Situation.

Copyright Hi, I'm Andrew. 2017
Tech Nerd theme designed by Siteturner