We are currently embarking on an Event Hub project where we will be processing device “reads” on a frequent basis. You can consider the use case to be in the Industrial IoT space but the difference is we do not have to manage the devices. Without geting into a lot of details, we have aggregators responsible for that. For this reason we decided not to pursue the Azure IoT Suite as we really just need Event Hub capabilities.
We will not have a lot of publishers, but the question did come up how can we protect, or restrict where traffic is coming from and how we manage keys to our service. Event Hubs currently does not have an ability to white list a set of IP Addresses. This implies that someone could take your SAS key and potentially ‘pollute’ your Event Hub from a location that has no business publishing to your Event Hub.
Another option is to issue SAS tokens, which do have expiration timestamps attached to them. This will not get you away from the location in which the publisher is pushing events to your Event Hub. But, it does ensure if you have a key leakage that its TTL (time to live) reduces risks. I am going to continue to explore this path in a later post or talk.
But for the purpose of this post, I am going to focus on API Managment (APIM) + Event Hubs to see what we can do. In a previous post, I did speak about using Azure API Management policies to limit where an API can be called from. We will leverage that post in the interest of keeping this post short(er).
The first thing we need to understand is the mechanics of calling an Event Hub using HTTP. The Service Bus team, does prefer AMQP and for good reasons (see Publishing an event section) but back to the whitelisting requirement we are left with HTTP. In order to call an Event Hub via HTTP there is some information that we need to collect including:
- Create ServiceBus Namespace
- Create Event Hub
- Create Shared Access Policy
In this case we will create a Send Access Policy. We do not want to provide uncessary privileges to our publisher; such as the ability to consume from our Event Hub.
- Generate a SAS token. To accomplish this feat, I used a tool by Sandrino Di Mattia which is available here: https://github.com/sandrinodimattia/RedDog/releases/tag/0.2.0.1 When you run the tool, you will be prompted for some of your ServiceBus information including Namespace, Event Hub Name, an arbitrary Publisher Name and details from the SAS Policy you just created. Once you provide this infromation and click on the Generate button you will have a SAS Token based upon the TTL that you specified. Copy this token as we will need it in future steps.
- We now can assemble our URL for our Event Hub based upon the following information:
|NAMESPACE:||Your Service Bus Namespace|
|EVENTHUB-NAME||Name of your Event Hub|
|PUBLISHER-NAME||This is somewhat arbitrary. What I used here was the name of my Shared Access Policy. Make sure to use the same value that you used to generate SAS Token.|
We can now test our ability to send a message to our Event Hub using HTTP and PostMan. In order to call our Event Hub using HTTP we will need our SAS token that we just generated and our URL.
In order to call the Event Hub endpoint we will need to create an Authorization Header and populate it with our SAS token.
Following that we can provide our URL and a message body that we want to send to Event Hubs. After clicking on the Send button we will see we get an HTTP 201 status code.
Azure API Management
We now know we can use HTTP to populate our Event Hub. With this done, lets configure our API Management instance. I am not going to go into a lot of detail on how to set up Azure API Management. I will refer you to a previous presentation video and tutorial on the subject.
Within Azure API Management we want to do the following
- Create an API In order to do this we will need our Event Hub URL that we used in PostMan.
- Create an Operation (and meta data). In this case there is no URL rewriting that needs to be performed. Our requests will be passed through.
- Apply our policies to Operation. In this case we will add two policies:
- IP Filter where I will include my IP Address and set action = “allow”
- Set Header where I will provide my SAS Token that will allow my API Management instance to talk to Event Hubs.
- Add API to Product
- Publish Product
- Assign Access to Product
Since we are using IP Whitelisting, I will make a call using PostMan, but this time I will send it through the API Management Proxy.
In order to call our endpoint through API Management, we will need our API Key which is available from the Azure Portal. We can access this information from the Users menu. We can then click on the Show link where we can then copy our key.
In PostMan, we will want to add an Authorization Header called ocp-apim-subscription-key and then provide our API Key that we just copied.
Next, we can provide our URL, message body and click Send button.
As you can see we have a successful publish through APIM and receive an HTTP 201 back.
To ensure our IP Whitelist is working, let’s try to call our service from the API Management Developer’s Console. As expected, we get an HTTP Status code of 403 Forbidden back which is as expected.