How System.Net.Http 4.3.0 Ruined Everyone's Day

I have not updated the MessageWire library for about a year now. But it still works just fine. Mostly. New Year's resolution #1: Update MessageWire.

trustme

Dependency Hell

I'm cleverly using MessageWire to create a customized shared session service for a hybrid set of web applications that include old ASP.NET Web Forms running in .NET Framework 4.6.1 and newer, upcoming, web sites built on ASP.NET Core 2.0. So first it had to work with the older Web Forms site. And it did work. Very well. I'll share some of that fun code on another day in another post.

What did not go well is that my Web Forms site using a library that uses HttpClient in System.Net.Http to call an internal web service suddenly failed.

Here's the HTTP request before pulling MessageWire into the project.

POST http://localhost:53739/api/ProfileDetail HTTP/1.1
Accept: application/json
Content-Type: application/json; charset=utf-8
Host: localhost:53739
Content-Length: 592

{"Id": "…removed actual data…"}

And here's the same request after pulling MessageWire into the project with no other code changes.

POST http://localhost:53739/api/ProfileDetail HTTP/1.1
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/json; charset=utf-8
Accept: application/json
Accept-Encoding: gzip, deflate
Host: localhost:53739
 
250
{"Id": "…removed actual data…"}
0

See the problem? Yikes, my web service was unable to deserialize the second request created with HttpClient v4.3.0, so of course, that led to a huge failure.

Everyone's Worst Nightmare

What's worst about this is that I did not know about this side effect and this code made it into production causing a nightmare during a critical moment of the day and the month. Of course we rolled it back to stop the bleeding, but damage was done and folks were not happy with me in the least. It took me the rest of the day to find the symptoms of the cause which precipitated the failure.

The fact that it took me so long felt like another failure. Even with the final discovery that the HTTP request was borked, I was clueless as to its exact cause. I even engaged several team members to pour through my code to find the problem with the code I had committed. They could not find it.

So today, with less pressure on, I began methodically thinking of it and tracing back the code that generates the HTTP request. Ultimately it came down to HttpClient in System.Net.Http. This led me to discover that pulling in MessageWire into my ASP.NET Web Forms application on .NET Framework 4.6.1 also pulled in the flawed System.Net.Http version 4.3.0 package.

Light bulb moment! Check for updates. Sure enough, there was a System.Net.Http 4.3.3 available. And boom goes the dynamite! The request was back to looking normal and working just fine with the service being called.

MS OSS Buyer Beware!

When you pull in any new NuGet package which brings with it some dependencies, be sure you check out any updates to those dependencies. Shocking as it may seem, even Microsoft's packages can contain nasty little bugs. And test your assumptions.

The Kids Make Our Christmas the Best

This Christmas has been a particularly joyful one. Despite one late evening lapse of patience, I can truly say this has been one of the most memorable celebrations of Christ's birth in my entire life. We were so pleased to have all of our children and two of our very good friends from India with us. Because of work schedules and other plans, we arranged to be visited by Santa Claus on the 24th and so today has become a quiet and peaceful reflective time. Of all the wonderful gifts I received this year, I want to share with you the one that touched me and my wife beyond any other.

Our children spent a day together on a recent weekend which they called "Sibling Day." It was not the first time this has happened. We are blessed with children who love each other and enjoy one another's company even now as adults. So Lorri and I suspected nothing. As we opened presents, going around taking turns as is our tradition, our children insisted that Lorri open one particular present last. I prevailed in having her open my gift last, but it paled in comparison to this one. It was a book and I've gathered their permission to share it here. I hope you'll indulge me then as I share with you a rather "social media" style personal post.

Tyler and Lorri

We have been married for thirty years. In that time we've seen a few ups and downs. I'm so grateful to have such a good wife by my side. She has been the anchor in my life against the tides and currents of changing and challenging times.

20171225_114631

This photo of Lorri and me was taken a few years ago by my sister Rebecca. She's a natural shutter bug.

20171225_114552

This one is the title page and was taken very recently. Both of us have lost a few pounds since the cover photo was taken. Lorri still looks younger and better than me. I'm the luckiest man alive.

The Kids Together

Jaelise, Johanna, John and Josh are four great kids. These re-enactment photos make us smile from ear to ear.

20171225_114522

20171225_114444

20171225_114410

20171225_114346

Josh

Our youngest and still living at home, Josh rarely had to be told to go to bed. Whenever he was upset with us, he would cover his head with a blanket or something and say, "You don't see me."

20171225_114309

20171225_114241

John

Our children were always fond of running around with no clothes. John was no exception. At four he learned to ride a bike and defied the rule of wearing a helmet, so he occasionally wore some road rash and once crashed hard enough to cause a severe concussion which had us in the ER for a while. Thankfully his skull was sufficiently strong and the damage was temporary. We think. (Just kidding, John.)

20171225_114210

20171225_114136

Johanna

The cutest messy eater in our house, Johanna was always perplexed when Lorri knew she had been sneaking some treat or another because the evidence was always on her face.

20171225_114000

20171225_113923

Jaelise

A ham for the camera then and now, Jaelise always loved to dress up and her bunny was a favorite. Her kayak has taken its place and goes nearly everywhere with her now.

20171225_113845

20171225_113758

Jenessa

Reading the book and laughing at each page, when Lorri came to this page that laughter turned to tears for both of us as we remembered our oldest and most perfect child. Jenessa lived just nine days and as we remembered the birth of Christ, these pages reminded us poignantly of our first child's birth. The elephant named Ernie was a small stuffed rainbow colored elephant who was buried with Jenessa in 1988. While we miss her so much, she is and always will be a very important part of our family.

20171225_113648

20171225_113619

My Kids Are The Best

When they say, "It's the thought that counts," I believe it. My children are thoughtful, loving and kind. I love them so very much. They have made this Christmas one of the very best of my life!

20171225_113546

Special thanks goes to Johanna who made all this happen. What an amazing gift!

Don't Have a Merry Christmas

I once thought it strange that the British say, "Happy Christmas," when I grew up in America saying, "Merry Christmas!" But after some reflection, I think the Brits have it right.

2 Nephi 28:7
"Yea, and there shall be many which shall say: Eat, drink, and be merry, for tomorrow we die; and it shall be well with us."

The merry making we do at Christmas time has little to do with the true meaning of Christmas. Parties, food, music, drink and all the trimmings hide the real purpose of Christmas and provide no lasting happiness.

christmasparty

I do not mean to say that having a party and making merry is necessarily bad. Simply that it is not the best way we can celebrate the Christ child and make others happy and be happy. Indeed some of my most happy memories of Christmas are of cold wintery mornings driving the tractor to plow the neighbor's long driveway or doing some other act of kindness for a neighbor or friend.

Perhaps my warmest memory of such a time was a particularly cold and wintry Christmas morning when my brother and I got up very early on Christmas morning and snuck out of the house to milk the cow, feed the animals and bring in buckets of coal to keep the house warm. These chores were normally done by our dad who allowed us to play with our gifts while he did all of our chores on Christmas morning. I still remember his smile and the look of surprise and love on his face when we told him he did not need to go out that morning.

You can learn a little about the history of Merry Christmas vs Happy Christmas here:

Perhaps we can all learn something about being happy this Christmas rather than just making merry. Let us try to light the world with His happiness.

I know that true joy, true happiness comes in and through our Lord and Savior Jesus Christ.

Matthew 25:40
And the King shall answer and say unto them, Verily I say unto you, Inasmuch as ye have done it unto one of the least of these my brethren, ye have done it unto me.

Let us remember him by remembering our neighbor this year. In other words, have a Very Happy Christmas!

Allow Your Models to Cross Only One Boundary

Don't let your models cross more than one boundary. Sure there may be exceptions, but I haven't found one that justifies the loss of flexibility achieved when requirements change.

Some years ago I worked with a thoughtful and intelligent programmer named Nick. He convinced me that models, or data transfer objects (DTOs) should not cross more than one service or domain boundary. I was skeptical at first. It seemed like a lot of duplicated code.

He talked me into it. And the next time the business changed a requirement, which of course happens often in most software development shops, I became absolutely convinced. We added something to one model, changed the code that produced the model and the specific code that consumed the model without breaking all other users of other endpoints that used the underlying models and entity models or services that produced them.

Here's what it looks like in one implementation:

models

Of course, your own architecture may involve more boundaries than shown here. You may have a message queue or services bus. I believe the principle still applies.

The greatest advantage is that you have an abstraction layer at every boundary that allows you to move swiftly when you need to support changing requirements. And I've never met a company or set of software developers that do not deal with regularly changing business requirements.

This is a very opinionated post and if you balk at the notion at first, I urge you to at least give it a try. You will find that you can enforce separation of concerns. You will find that you can avoid having a dependency on a framework, such as Entity Framework, in your application server code or in your hard wired client code, the latter of which would need only a reference to your ViewModels assembly. (Assumes you are working in .NET but I believe the principles apply universally.)

You will find that in preventing your business logic from knowing anything about the implementation of the interface provided by the Models assembly, you are forced to inject the dependency and are thus free to replace the dependency with a different implementation or even a mock for unit testing.

The list of benefits is long and I've surely missed a few important ones, but you get my drift. I hope.

TIP: Use AutoMapper or something like it to help you with the tedium when transformations are an exact 1:1 match.

Getting Back on the Horse

I grew up on a farm and ranch driving tractors mostly but riding horses to herd cows and to play as often as was possible. It was an idyllic way to grow up. I was more of a farmer than I was a cowboy, but I loved old western movies, especially John Wayne, and idolized the American Cowboy and the cowboy way of life.

This painting by a family friend named Lynn E. Mecham reminds me of all that I love about cowboys.

cowboys_thumb2

I encourage you to visit his Facebook page for his gallery. He does amazing work.

As a young boy one of my favorite pastimes was to saddle up the horse and join a few of my friends in a game of tag on horseback in the cedar and sage brush hills above the small farm on which I grew up. Of course we knew nothing really about the politics surrounding the treatment of Native Americans. For us it was just a game.

On one of these occasions, I was riding my brother's mare Toots, a beautiful buckskin that looked very much like this.

toots_thumb1

I was an Indian. One of my friends was selected to be the Cowboy, the guy who had to catch one of us. I had Toots running through the sage brush and planned to ditch the Cowboy in a barrel turn around a large cedar tree coming up on my left. I pulled left on the reins at just the right moment and she juked left around the tree.

Suddenly I was laying on the sand on my back, pain throbbing in my chest and, for a moment, I was unable to breathe. The tree had a large branch sticking out right at my chest height from the saddle. I was not hurt badly. I found Toots looking back at me as if to say, "What happened to you?" I got back on after a few minutes and we continued playing.

I learned a valuable lesson that day. Even when you're in a hurry, be cautious when rushing into a place you've never before been. You might get knocked off your horse, but you'll probably survive.

Fast forward a few decades to this year. After surviving a health crisis in India, I'm now dressing like a cowboy to remind myself to work hard and take better care of myself. And I'm still "playing" with Indians. But in this case real Indians.

I work with two teams of wonderful developers and testers in the New Delhi area. I have spent a little more than a year working with them. I was enjoying myself so much that I forgot the lesson of my youth and traveled in July to work and have fun with my Indian colleagues.

funday2_thumb1

I nearly died in India after that fun day. I got knocked off my horse. It took several weeks to get back up, but I did. And I may come off my horse again, but like my dad always taught me, I will get back on. I hope you do too.

5 Keys to JSON Web Tokens and ASP.NET Core 2

I know JSON Web Tokens (JWT) have been around for a while, but I still run into many developers who do not know much or anything about them. You can read much more in the JWT Handbook and at one of my favorite online tool sites for using JWT at jwt.io.

Here are the 5 things I think you should know about JWT:

#1. OpenID

If you're not already using OpenID, a standard for single sign-on and identity provision, you should be looking into it. There are many sites where you can learn more about OpenID. The connect2id.com OpenID overview is a good place to start. And if you're using ASP.NET Core 2 and IdentityServer4, you definitely want to know more about JWT and OpenID.

#2. Security Concerns

You can encrypt your JWT tokens but most do not. You can include whatever you want in terms of claims in the token. Here's an example of a token trimmed down for length to simply demonstrate what an encoded token may look like. Note the "." dividers.

eyJ0eXAiOF1RSJ9.eyJpc3MiOiJNlcyIsImV4cb3JkIl19.AsYr7vaxFqCztZ_wph_

Here is what that token may look like when decoded from the Base 64 string:

Header

{
  "typ": "JWT",
  "alg": "RS256",
  "x5t": "a3rMUgMFv9tPclLa6yF3zAkfquE",
  "kid": "a3rMUgMFv9tPclLa6yF3zAkfquE"
}

Payload Data

{
  "iss": "https://identity.yourdomain.com/identity",
  "aud": "https://identity.yourdomain.com/identity/resources",
  "exp": 1512842905,
  "nbf": 1512839305,
  "client_id": "apiclient",
  "scope": "openid",
  "sub": "108434458",
  "auth_time": 1512839305,
  "idp": "youridpid",
  "amr": [
    "password"
  ]
}

So what do you need to be worried about with respect to security? Simply this. Make sure whatever you put into your JWT tokens do not reveal secrets or sensitive data that you do not want discovered. Yes, of course, the JWT is signed. The third part of the token is a hash created by the identity server and must be validated on the server side accepting the token when you use it in the Authorization header as a Bearer token.

This means the server side has to have the key used to validate. But on the client side, for unencrypted JWT tokens, the data can be useful. Just don't make it useful to the bad actor. And when you decide to add claims to your JWT from your identity server implementation, be sure you are comfortable with that data being out in the wild.

#3 Authorization Most Common Use

The most common use of a JWT is as a bearer token in the Authorization header as I've just mentioned.

Authorization: Bearer [token]

Here's the flow.

  1. Browser (or API client) sends POST to identity endpoint with username and password.
  2. Identity server authenticates and creates JWT signed with secret key or certificate.
  3. Browser (or API client) receives the JWT token.
  4. Browser (or API client) makes request to the app server with Authorization header with "Bearer [token]" in request.
  5. App server validates the JWT token and authorizes and executes the request or denies it.
  6. Browser (or API client) receives response from the application server.

#4 Getting the "sub" Claim Back from Microsoft's Mapping

Before you spend a lot of time in ASP.NET Core 2 trying to get access to the "sub" (subject) claim in the controller's User (cast as ClaimsPrincipal), just know that it's not your fault. It's Microsoft's. They map claims and are not 100% compliant with the OpenID standard in that the "sub" claim disappears. Here's how you get it back:

public void ConfigureServices(IServiceCollection services)
{
  //config services like logging, Mvc, etc.
  
  JwtSecurityTokenHandler.DefaultInboundClaimTypeMap.Remove("sub"); 
  
  //config services.AddAuthentication and AddAuthorization
}

By removing the "sub" claim from their claim type map, it will reappear in your ClaimsPrincipal like this dummy code written simply to show how to grab the "sub" claim now that it's back.

[HttpGet("{id}"), Authorize("openid")]
public string Get(int id)
{
  var user = User as ClaimsPrincipal;
  var memId = user.Claims
      .Where(x => x.Type == "sub")
      .Select(x => x.Value).FirstOrDefault();
  return memId;
}

Try that without removing it from the type map and you'll get a big fat null.

#5 JWT Tokens Can and Should Expire

If you are using JWT and OpenID, you should make sure that your identity server sets an expiration on the token. And if your authentication request provides your with a refresh token, you should learn how in the client to refresh (call the identity server with the refresh token) your access JWT token. See auth0's details on using refresh tokens.

A refresh token is especially useful for keeping a user logged in without asking her or him to enter a username and password again. And it is a great way to avoid storing credentials locally. Yeah, don't do that.

Conclusion

If you're not using JWT or have not yet become comfortable with tokens, you owe it to yourself to do a little reading, even if you're not a coder. But if you're a coder, you definitely need to grok JWT.

I'm Glad I Died in India

Last summer I died in India, figuratively if not quite literally. And I'm glad..

I won't bore you with the details. Here's the nutshell. My wife and I traveled to New Delhi for work and fun to be followed a 9 day 30th anniversary celebration in Ireland on our way home. A health crisis intervened. I nearly died twice. I spent ten days in ICU under the care of the very capable physicians and other professionals at Max Hospital in New Delhi.

I lost more than just 25 pounds and 15 more since then. The person I was died there in that hospital bed. I returned with a renewed determination to waste no more time on the irrelevant and the unnecessary. I left behind a man of too much wasted time and too many unproductive habits. I came home with a new lease on life, and I wasn't going to waste it.

newmeFirst and foremost, thanks go to my dear wife without whom I certainly would have died. Secondly great thanks go to all my Indian coworkers, several of whom participated in my rescue from certain death, and the management team back home who made sure that our every need was very well met.

This photo was taken of me a few weeks after our return to Utah. It's how I dressed as a young kid growing up on a farm and ranch. It's how my dad still dresses. It's my way of reminding myself to be more like him and focus on the important things in life while eliminating things that don't matter and just waste time.

Because my health crisis was in precipitated by my poor personal health habits of the past and a nasty intestinal infection contracted in my first few days in country, I've spent a good deal of time learning how to take better care of myself with a sensible diet and exercise, making good on a commitment to create a lifelong habit and avoid repeating my nightmare in New Delhi.

I stopped watching television. I stopped paying much attention to politics and I never waste time commenting on that topic. I spend more time with my family. I rise early and retire early. I spend time in personal study and devotions every day. And I strive to treat others, especially those with whom I work, as I would wish to be treated, with kindness and patience.

It has been a transformative experience. I grew so much closer to the great people on my teams and all those who took care of my wife while I fought off death in the hospital. I know I'm not the only one to have gone through something like this. There are many who emerge from such a health crisis changed, better, more alive than ever before. I am deeply grateful for the opportunity to see from this perspective, though I recommend you take my word for it. And it sure beats looking up from six feet under.

...

P.S. It's been nearly a year since I posted my last blog post. You can expect to see me posts more often. Most of my posts in this blog have been about software development. I'm thinking you'll see a mix of technology and personal posts in the future. I hope you don't mind.