#Integrate2017

Integrate is once again upon us and I am happy to announce that I will once again be speaking at it.  The event occurs from June 26 – 28th at Kings Place, London.  This is my 4th trip to London to speak at the event and I am very excited to be heading back to Europe.  Thank-you BizTalk360 for the opportunity to speak at this conference again.

image

Early bird pricing is still in effect until the end of May and seats are going fast. Please visit the event website for more details.

This is the top Microsoft Integration conference and as a result the bar is pretty high for content. As a result, I have a confession.  I have been holding back on #MiddlewareFriday.

Image result for confession

Every week I try to come up with something interesting to talk about, but I wanted to ensure I had an especially interesting topic to share in London. So behind closed doors, I have been building my presentation and demos over the past few months. The result is a talk called Give your Bots connectivity, with Azure Logic Apps.
At my employer, I have been thinking a lot of what the future of the workforce looks like and the impact that Artificial Intelligence (AI) will have on organizations.  One of the areas that I have been exploring within AI is BOTS, or conversational apps.  I think these software applications are very interesting and have the opportunity to improve end user experiences and flush out manual work from organizations.
However, a BOT is pretty much useless without connectivity.  Azure Logic Apps makes a great companion for BOTS as we execute tasks across a variety of cloud and on-premises systems.
We will also explore some Azure Cognitive Services that will give our BOT more human-like intelligence in order to reduce friction between our BOT and user.

See you there!!!

#MiddlewareFriday

The purpose of this post is to talk about a side-project that I have going on with Saravana Kumar and BizTalk360. The purpose of #MiddlewareFriday is to create a video blog of new and interesting developments going on in the industry.  Each week we will publish a short video that has some new content.  The content may feature news, demos and will also highlight other activities going on in the community.  From time to time we will also bring on some guests to keep the content fresh and get some different perspectives.

For both Saravana and myself there is no direct commercial incentive in doing the show.  It really comes down to participating in a community, learning by doing, improving communication skills and having some fun along the way.

I am going to keep this post updated to keep a running list of the shows – in part to aid in search engine discovery.

Episode Title Date Tags
1 Protecting Azure Logic Apps with Azure API Management January 6, 2017 Azure API Management, Logic Apps, ServiceNow, API Apps
2

Azure Logic Apps and Service Bus Peek-Lock

January 13, 2017 Logic Apps, Service Bus, Patterns
3

Logic Apps and Cognitive Services Face API – Part 1

January 20, 2017 Logic Apps, Cognitive Service, Face API, Steef-Jan Wiggers
4 Microsoft PowerApps and Cognitive Services Face API -Part 2 January 27, 2017 PowerApps, Cognitive Services, Face API
5 Serverless Integration with Steef-Jan Wiggers February 3rd, 2017 Logic Apps, Sentiment Analysis, Slack, Azure Functions, Steef-Jan Wiggers
6 Azure Logic Apps and Power BI Real Time Data Sets February, 10, 2017 Logic Apps, Power BI connector, Sandro’s Integration stencils, Quicklearn, Global Integration Bootcamp
7 Azure Monitoring, Azure Logic Apps and Creating ServiceNow Incidents February 17, 2017 Logic Apps, Azure Monitoring, API Apps, ServiceNow, Glen Colpaert SAP, Webhook Notification BizTalk360
8 Exploring ServiceBus360 Monitoring February 24, 2017 Service Bus, BizTalk360, Community Content: Team Flow + Luis, Exception handling for Logic App Web Services Toon Vanhoutte
9 SAP and Logic Apps (Part 1) March 3rd, 2017 Logic Apps, On-premises data gateway, SAP, Steef-Jan Wiggers, Michael Stephenson, Logic Apps Live review
10 SAP and Logic Apps (Part 2) using Enterprise Integration Pack March 10, 2017 Integration Pack, SAP – Richard Seroter, Enterprise Architecture for cloud natives, BizTalk 2016 Task Schedule Adapter
11 Logic Apps – Inbound Custom HTTP Paths/Methods March 17th, 2017 HTTP Request, Custom Paths, GET, DELETE, Johan Hedberg, Continuous Integration
12 Using Azure API Management to protect BizTalk Server Endpoints – with Steef-Jan Wiggers March 24th, 2017 Azure API Management, BizTalk, Azure Service Bus Relay
13 Global Integration Bootcamp – Highlights from New York City March 31st, 2017 Global Integration Bootcamp, Stephen W Thomas, Howard Edidin, Mandi Ohlinger
14 Azure Functions, Swagger and API Management April 7th, 2017 APIM, Azure Functions, Global Integration Bootcamp, Wagner Silveira
15 Introduction to Azure Functions Proxies April 14th, 2017 Azure Functions, Hybrid Integration Sandro Pereira, APIM, Tord Glad Nordahl
16 Azure Logic Apps and Azure Event Hubs April 21th, 2017 Azure Stream Analytics, Santosh Balasubramanian, Event Hubs Logic Apps
17 BizTalk Server 2016 – First Look April 28th, 2017 BizTalk Server, Tips and Tricks Logic Apps Cost Savings, Rene Brauwers, Boston Marathon Experience
18 BizTalk Server 2016 + Logic Apps – Thunder and Lightning May 5th, 2017 BizTalk Server 2016, Logic Apps, Azure Functions, Michael Stephenson
19 BizTalk Server 2016 – Fabulous Pack 1 May 12, 2017 BizTalk Server Feature Pack 1, Installation, Continuous Deployment, Sandro Pereira, Toon Vanhoutte
20 Task Automation Showdown featuring Azure Logic Apps and Flow (Coming Soon) by Steef-Jan Wiggers May 19th, 2017 Steef-Jan Wiggers

2016 Year in Review, Looking Ahead to 2017

It is that time of year where I like to reflect back on what the previous year has brought and also set my bearings for the road ahead.  If you are interested in reading my 2015 recap, you can find it here.

Personal

2016 was a milestone birthdate for myself and my twin brother. In order to celebrate, and try to deny getting old for another year, we decided to run a marathon in New York City.  The NYC Marathon is one of the 6 major marathons in the world so it acted as a fantastic backdrop for our celebration.  Never one to turn down an adventure, my good friend Steef-Jan Wiggers joined us for this event.  As you may recall, Steef-Jan and I ran the Berlin Marathon (another major) back in 2013.

Marathon

The course was pretty tough.  The long arching bridges created some challenges for me, but I fought through it and completed the race.  We all finished within about 10 minutes of each and had a great experience touring the city.

Before Race 

Kurt, Kent and Steef-Jan in the TCS tent before the race

AfterRace

At the finish line with the hardware.

After Pary

Celebrating our victory at Tavern on the Green in Central Park.

Speaking

Traveling and speaking is something I really like to do and the MVP program has given me many opportunities to scratch this itch. I also need to thank my previous boss and mentor Nipa Chakravarti for all of the support that she has provided which made all of this possible.

In Q2, I once again had a chance to head to Europe to speak at BizTalk360’s Integrate Event with the Microsoft Product Group.  My topic was on Industrial IoT and some of the project work that we had been working on. You can find a recording of this talk here.

Speaking

On stage….

Integrate

I really like this photo as it reminds me of the conversation I was having with Sandro.  He was trying to sell me a copy of his book, and I was trying to convince him that if he gave me a free copy, that I could help him sell more.  Sandro has to be one of the hardest working MVPs I know who is recognized as one of the top Microsoft Integration Gurus.  If you have ever having a problem in BizTalk, there is a good chance he has already solved it.  You can find his book here in both digital and physical versions.

Integrate Group

BizTalk360 continues to be an integral part of the Microsoft Integration community.  Their 2016 event had record attendance from more than 20 countries.  Thank-you BizTalk360 for another great event and for building a great product.  We use BizTalk360 everyday to monitor our BizTalk and Azure services. 

On a bit of a different note, this past year we had a new set of auditors come in for SOX compliance.  For the first time, that I have experienced, the auditors were really interested in how we were monitoring our interfaces and what our governance model was.  We passed the audit with flying colours, but that was really related to having BizTalk360.  Without it, our results would not have been what they were.

Q3

Things really started to heat up in Q3.  My first, of many trips, was out to Toronto to speak at Microsoft Canada’s Annual General meeting. I shared the stage with Microsoft Canada VP Chris Barry as we chatted about Digital Transformation and discuss our experiences with moving workloads to the cloud.

AGM

Next up was heading to the south east United States to participate in the BizTalk Bootcamp. This is my third time presenting at the event.  I really enjoy speaking at this event as it is very well run and is in a very intimate setting.  I have had the chance to meet some really passionate integration folks at this meetup so it was great to catch up once again.  Thank-you Mandi Ohlinger and the Microsoft Pro Integration team for having me out in Charlotte once again.

BizTalkBootcamp

At the Bootcamp talking about Azure Stream Analytics Windowing.

The following week, I was off to Atlanta to speak at Microsoft Ignite.  Speaking at a Microsoft premier conference like Ignite (formerly TechEd) has been a bucket list item so this was a really great opportunity for me.  At Ignite, I was lucky enough to participate in two sessions.  The first session that I was involved in was a customer segment as part of the PowerApps session with Frank Weigel and Kees Hertogh.  During this session I had the opportunity to show off one of the apps my team has built using PowerApps.  This app was also featured as part of a case study here.

Powerapps

On stage with PowerApps team.

Next up, was a presentation with John Taubensee of the Azure Messaging team.  Once again my presentation focused on some Cloud Messaging work that we had completed earlier in the year.  Working with the Service Bus team has been fantastic this year.  The team has been very open to our feedback and has helped validate different use cases that we have.  In addition to this presentation, I also had the opportunity to work on a customer case study with them.  You can find that document here. Thanks Dan Rosanova, John Taubensee, Clemens Vasters and Joe Sherman for all the support over the past year.

Ignite

Lastly, at the MVP Summit in November, I had the opportunity to record a segment in the Channel 9 studio.  Having watched countless videos on Channel 9, this is always a neat experience.  The segment is not public yet, but I will be sure to post when it is.  Once again, I had the opportunity to hang out with Sandro Pereira before our recordings.

Sandro

Channel9Studio

In the booth, recording.

Channel9StudioCouch

Prepping in the Channel 9 studio

Writing

I continue to write for InfoQ on Richard Seroter’s Cloud Editorial team.  It has been a great experience writing as part of this team.  Not only do I get exposed to some really smart people, I get exposed to a lot of interesting topics that only fuels my career growth.  In total, I wrote 46 articles but here are my top 5 that I either really enjoyed writing or learned a tremendous amount about.

  • Integration Platform as a Service (iPaaS) Virtual Panel In this article, I had the opportunity to interview some thought leaders in the iPaaS space from some industry leading organizations.  Thank-you Jim Harrer (Microsoft), Dan Diephouse (MuleSoft) and Darren Cunningham (SnapLogic) for taking the time to contribute to this feature.  I hope to run another panel in 2017 to gauge how far iPaaS has come.
  • Building Conversational and Text Interfaces Using Amazon Lex – After researching this topic, I immediately became interested in Bots and Deep Learning.  It was really this article that acted as a catalyst for spending more time in this space and writing about Google and Microsoft’s offerings.
  • Azure Functions Reach General AvailabilitySomething that I like to do, when possible, is to get a few sound bytes from people involved in the piece of news that I am covering.  I met Chris Anderson at the Integrate event earlier in the year, so it was great to get more of his perspective when writing this article.
  • Microsoft PowerApps Reaches General Availability – Another opportunity to interview someone directly involved in the news itself.  This time it was Kees Hertogh, a Senior Director of Product Marketing at Microsoft. 
  • Netflix Cloud Migration Complete – Everyone in the industry knows that Netflix is a very innovative company and has disrupted and captured markets from large incumbents.  I found it interesting to get more insight into how they have accomplished this.  Many people probably thought the journey was very short, but what I found was that it wasn’t the case.  It was a very methodical approach that actually took around 8 years to complete.

Another article that I enjoyed writing was for the Microsoft MVP blog called Technical Tuesday.  My topic focused on Extending Azure Logic Apps using Azure Functions. The article was well received and I will have another Technical Tuesday article published early in the new year.

Back to School

Blockchain

I left this topic off of the top 5 deliberately as I will talk about it here, but it absolutely belongs up there. Back in June, I covered a topic for InfoQ called Microsoft Introduces Project Bletchley: A Modular Blockchain Fabric.  I really picked up this topic out of our Cloud queue as my boss at the time had asked me about Blockchain and I didn’t really have a good answer. After researching and writing about the topic, I had the opportunity to attend a Microsoft presentation in Toronto for Financial organizations looking to understand Blockchain.  At the Microsoft event (you can find similar talk here), Alex Tapscott gave a presentation about Blockchain and where he saw it heading.  ConsenSys, a Microsoft partner and Blockchain thought leader was also there talking about the Brooklyn Microgrid. I remember walking out the venue that day thinking everything was about to change.  And it did.  I needed to better understand blockchain.

For those that are not familiar with blockchain, simply put, it is a paradigm that focuses on using a distributed ledger for recording transactions and providing the ability to execute smart contracts against these transactions.  An underlying principle of blockchain is to address the transfer of trust amongst different parties.  Historically, this has been achieved through intermediaries that act as a “middleman” between trading partners.  In return, the intermediary takes a cut on the transaction, but doesn’t really add a lot of value beyond collecting and dispersing funds.  Trading parties are then left to deal with the terms that the intermediary sets.  Using this model typically does not provide incentives for innovation, in fact it typically does the opposite and stifles it due to complacency and entitlement by large incumbent organizations.

What you will quickly discover with blockchain is that it is more about business than technology.  While technology plays a very significant role in blockchain, if your conversation starts off with technology, you are headed in the wrong direction.  With this in mind, I read Blockchain Revolution by Alex and Don Tapscott which really focuses on the art of the possible and identifying some modern-day scenarios that can benefit from blockchain.  While some of the content is very aspirational, it does set the tone for what blockchain could become.

Having completed the book, I decided to continue down the learning path.  I wanted to now focus on the technical path.  I am a firm believer that in order for me to truly understand something, I need to touch it.  By taking the Blockchain Developer course from B9Lab I was able to get some hands on experience with the technology.  As a person that spends a lot of time in the Microsoft ecosystem, this was a good learning opportunity to get back into Linux and more of the open source community as blockchain tools and platforms are pretty much all open source.  Another technical course that I took was the following course on Udemy.  The price point for this course is much lower, so it may be a good place to start without making a more significant financial investment in a longer course.

Next, I wanted to be able to apply some of my learnings.  I found the Future Commerce certificate course from MIT.  It was a three month course, all delivered online.  There were about 1000 students, worldwide, in the course and it was very structured and based upon a lot of group work.  I had a great group that I worked with on an Energy-based blockchain startup.  We had to come up with a business plan, pitch deck, solution architecture and go to market strategy, Having never been involved in a start-up at this level (I did work for MuleSoft, but they were at more than 300 people at the time), it was a great experience to work through this under the tutelage of MIT instructors. 

If you are interested in the future of finance, aka FinTech, I highly recommend this course.  There is a great mix of Finance, Technology, Entrepreneurs, Risk and Legal folks in this class you will learn a lot.

MIT

Gary Vaynerchuk

While some people feel that Twitter is losing its relevancy, I still get tremendous value out of the platform.  The following is just an example.  Someone I follow on Twitter is Dona Sarkar, from Microsoft, I had the opportunity to see her speak at the Microsoft World Partner Conference and quickly became a fan.  Back in October, she put out the following tweet, which required further investigation on my part.

Gary V

Dona’s talks, from the ones that I have seen, are very engaging and also entertaining at the same time.  If she is talking about “Gary Vee” in this manner, I am thinking there is something here.  So I start to digest some of his content.  I was very quickly impressed.  What I like about Gary is he has a bias for action.  Unfortunately, I don’t see this too often in Enterprise IT shops; we try to boil the ocean and watch initiatives fail because people have added so much baggage that the solution is unachievable or people have become disenfranchised.  I have also seen people being rewarded for building “strategies” without a clue how to actual implement them.  I find this really prevalent in Enterprise Architecture where some take pride in not getting into the details.  While you may not need to stay in the details for long, without understanding the mechanics, a strategy is just a document.  And a strategy that has not/cannot be executed is useless.

If you have not spent time listening to Gary, here are some of his quotes that really resonated with me.

  • Bet on your strengths and don’t give a f&%# about what you are not good at.
  • Educate…then Execute
  • You didn’t grow up driving, but somehow you figured it out.
  • Results are results are results
  • I am just not built, to have it dictate my one at-bat at life.
  • Document, Don’t Create.
  • We will have people who are romantic and hold onto the old world who die and we will have people that execute and story tell on the new platform who emerge as leaders in the new world.
  • I am built to get punched in the mouth, I am going spit my front tooth out and look right back at you and be like now what bitch.

image

If this sounds interesting to you, check out a few of his content clips that I have really enjoyed:

Looking Forward

I find it is harder and harder to do this.  The world is changing so fast, why would anyone want to tie themselves down to an arbitrary list? Looking back on my recap from last year, you won’t find blockchain or bots anywhere in that post, yet those are two of the transformative topics that really interested me in 2016.  But, there are some constants that I don’t see changing.  I will continue to be involved in the Microsoft Integration community, developing content, really focused on iPaaS and API Management.  IoT continues to be really important for us at work so I am sure I will continue to watch that space closely.  In fact, I will be speaking about IoT at the next Azure Meetup in Calgary on January 10th.  More details here.

I will also be focusing on blockchain and bots/artificial intelligence as I see a lot of potential in these spaces.  One thing you can bet on is that I will be watching the markets closely and looking for opportunities where I see a technology disrupting or transforming incumbent business models.

Also, it looks like I will be running a marathon again in 2017.  My training has begun and am just awaiting confirmation into the race.

The Current State of iPaaS

In addition to my day job of being an Enterprise Architect at an Energy company in Calgary, I also write for InfoQ.  Writing for InfoQ allows me to explore many areas of Cloud and engage with many thought leaders in the business.  Recently, I had the opportunity to host a Virtual Panel on Integration Platform as a Service (iPaaS).

Participating in this panel was:

  • Dan Diephouse – Director of Product Management at MuleSoft where he is part of the team that launched MuleSoft’s iPaaS offering: CloudHub.
  • Darren Cunningham – Vice President of Marketing at SnapLogic where he focuses on product management and outbound product marketing.
  • Jim Harrer – Principal Group Program Manager in the Cloud & Enterprise division, at Microsoft, where his team is responsible for Program Management for BizTalk Server and Microsoft’s iPaaS offering: Azure Logic Apps.

Overall, I was very happy with the outcome of the article.  I think the panelists offered some great insight into the current state of iPaaS and where this paradigm is headed.

You can read the entire article here and feel free to add comments in the article’s comment section.

Protecting your Azure Event Hub using Azure API Managment

We are currently embarking on an Event Hub project where we will be processing device “reads” on a frequent basis. You can consider the use case to be in the Industrial IoT space but the difference is we do not have to manage the devices.  Without geting into a lot of details, we have aggregators responsible for that.  For this reason we decided not to pursue the Azure IoT Suite as we really just need Event Hub capabilities.

We will not have a lot of publishers, but the question did come up how can we protect, or restrict where traffic is coming from and how we manage keys to our service.  Event Hubs currently does not have an ability to white list a set of IP Addresses.  This implies that someone could take your SAS key and potentially ‘pollute’ your Event Hub from a location that has no business publishing to your Event Hub.

Another option is to issue SAS tokens, which do have expiration timestamps attached to them. This will not get you away from the location in which the publisher is pushing events to your Event Hub.  But, it does ensure if you have a key leakage that its TTL (time to live) reduces risks.  I am going to continue to explore this path in a later post or talk.

But for the purpose of this post, I am going to focus on API Managment (APIM) + Event Hubs to see what we can do.  In a previous post, I did speak about using Azure API Management policies to limit where an API can be called from.  We will leverage that post in the interest of keeping this post short(er).

The first thing we need to understand is the mechanics of calling an Event Hub using HTTP.  The Service Bus team, does prefer AMQP and for good reasons (see Publishing an event section) but back to the whitelisting requirement we are left with HTTP.  In order to call an Event Hub via HTTP there is some information that we need to collect including:

  • Create ServiceBus Namespace
  • Create Event Hub
  • Create Shared Access Policy

    In this case we will create a Send Access Policy.  We do not want to provide uncessary privileges to our publisher; such as the ability to consume from our Event Hub.

    image

  • Generate a SAS token. To accomplish this feat, I used a tool by Sandrino Di Mattia which is available here:  https://github.com/sandrinodimattia/RedDog/releases/tag/0.2.0.1  When you run the tool, you will be prompted for some of your ServiceBus information including Namespace, Event Hub Name, an arbitrary Publisher Name and details from the SAS Policy you just created.  Once you provide this infromation and click on the Generate button you will have a SAS Token based upon the TTL that you specified. Copy this token as we will need it in future steps.image
  • We now can assemble our URL for our Event Hub based upon the following information:
URL:  https://NAMESPACE.servicebus.windows.net/EVENTHUB-NAME/publishers/PUBLISHER-NAME/messages
NAMESPACE: Your Service Bus Namespace
EVENTHUB-NAME Name of your Event Hub
PUBLISHER-NAME This is somewhat arbitrary.  What I used here was the name of my Shared Access Policy.  Make sure to use the same value that you used to generate SAS Token.

TESTING

We can now test our ability to send a message to our Event Hub using HTTP and PostMan.  In order to call our Event Hub using HTTP we will need our SAS token that we just generated and our URL.

In order to call the Event Hub endpoint we will need to create an Authorization Header and populate it with our SAS token.

image

Following that we can provide our URL and a message body that we want to send to Event Hubs. After clicking on the Send button we will see we get an HTTP 201 status code.

image

Azure API Management

We now know we can use HTTP to populate our Event Hub.  With this done, lets configure our API Management instance.  I am not going to go into a lot of detail on how to set up Azure API Management.  I will refer you to a previous presentation video and tutorial on the subject.

Within Azure API Management we want to do the following

  • Create an API In order to do this we will need our Event Hub URL that we used in PostMan.

image

  • Create an Operation (and meta data).  In this case there is no URL rewriting that needs to be performed.  Our requests will be passed through.

image

  • Apply our policies to Operation.  In this case we will add two policies:
    • IP Filter where I will include my IP Address and set action = “allow”
    • Set Header where I will provide my SAS Token that will allow my API Management instance to talk to Event Hubs.

image

  • Create Product

    image

    • Add API to Product

    image

    • Publish Product
    • Assign Access to Product

      image

      Testing

      Since we are using IP Whitelisting, I will make a call using PostMan, but this time I will send it through the API Management Proxy.

      In order to call our endpoint through API Management, we will need our API Key which is available from the Azure Portal.  We can access this information from the Users menu.  We can then click on the Show link where we can then copy our key.

      image

      In PostMan, we will want to add an Authorization Header called ocp-apim-subscription-key and then provide our API Key that we just copied.

      image

      Next, we can provide our URL, message body and click Send button.

      image

      As you can see we have a successful publish through APIM and receive an HTTP 201 back.

      image

      To ensure our IP Whitelist is working, let’s try to call our service from the API Management Developer’s Console. As expected, we get an HTTP Status code of 403 Forbidden back which is as expected.

      image

      Slide Deck from Prairie Dev Con Session

      Thank you to those of you who attended my “Introduction to Microsoft’s Middleware in the Cloud” session yesterday at PrairieDevCon West in Calgary.  I appreciated the level of engagement and the amount of questions that were asked.  Below you will find a link to the Slide Deck you saw yesterday.  I have also added a few slides to include screen shots of the demo applications that you saw to provide some additional context.

      https://skydrive.live.com/redir.aspx?cid=db51ef47e2bb432a&resid=DB51EF47E2BB432A!5663&parid=DB51EF47E2BB432A!375&authkey=!AOe1Rn7b80mPYdg  (Open in Powerpoint for Animations)

      New Blog Coming Soon

      Hi,

      New blog is coming soon to this address. This blog will be dedicated to Middleware in the cloud (as the names suggests) and will focus more so on Microsoft’s offerings including AppFabric Service Bus, AppFabric Applications, Queues, Topics and any other interesting bits coming out of Redmond.

      In the mean time you can visit my BizTalk blog @ http://kentweare.blogspot.com

      Thanks,

       

      Kent