Jump to content
Sign in to follow this  
Guest sky.sanders

JsonClient Design Patterns (working title)

Recommended Posts

Guest sky.sanders
(this should really be a threaded forum discussion with the conclusions related here with links, but this is what we have for now)

 


The JsonClient design consists of 3 primary components,

 


  • IJsonClient - presents the callable surface area of the library
  • IRequestCache - a map of CityIndex.JsonClient.ICacheItem keyed by url 
  • IThrottedRequestQueue - a queue of CityIndex.JsonClient.IRequestHolder that is processed, on a timer, at the rate specified by the threshold fields.

Share this post


Link to post
Guest sky.sanders

JsonClient

CityIndex.JsonClient.IJsonClient

 represents the core of the library and exposes 3 methods (with overloads for brevity)

  • BeginRequest<TDTO>
  • EndRequest<TDTO>
  • Request<TDTO>

The first two are obviously an async implementation and the last, 

Request<TDTO>

 (a better name is welcome), presents itself as a sync method but is, internally, a wrapper that calls the async methods.

I chose this route to elimate logic duplication that would be introduced if I were to leverage the sync methods of WebRequest. I made this mistake in Soapi and have never stopped regretting it.

BeginRequest<TDTO>/EndRequest<TDTO>

 present a standard .net async pattern in that you call 

BeginRequest

 with an async callback and the relevant parameters and receive an async result object in the callback.

EndRequest

 is called with the async result object and the result of the request is returned OR any exception encountered during the execution of the request is rethrown. This functionality could be exposed directly on the async result but following the standard .net async pattern allows for a checkpoint in JsonClient for metrics and/or corelation.

...
client.BeginRequest<FooDTO>(MyCallback, state,params);
...     
void MyCallback(ApiAsyncResult<FooDTO> result)
{
      try
   {
      FooDTO response = client.EndRequest(result);
      // do something with response.
   }
   catch(Exception ex)
   {
      // something bad happened
   }
}
Request<TDTO>

, as stated, is a simple blocking wrapper for the async implementation.

...
try
{
   FooDTO response = client.Request<FooDTO>(params);
   // do something with response.
}
catch(Exception ex)
{
   // something bad happened
}

Thus concludes the public surface area of the functionality of the JsonClient library.

Share this post


Link to post
Guest sky.sanders

Caching

Internally, when JsonClient begins a request it locks and checks the internal cache for an existing (or pending) response (CityIndex.JsonClient.CacheItem) with a matching url.

A CacheItem may be in one of several states as enumerated in CacheItemState. Explaination of various states and the action taken in BeginRequest follows:

New

The item was created and inserted by the cache and is ready to be populated.

This will typically happen when no matching, non-expired request was found in the cache.

The cache query method, GetOrCreate, in the interest of atomicity, creates, inserts and returns a new CacheItem when none is found to match the url.

NOTE: This is the only case in which an actual WebRequest is dispatched to the throttle.

Pending

The item’s request has been issued and is waiting for completion.

While in this state, additional callbacks may be added for identical requests enabling a single request to service multiple calls.

Processing

The item’s request has completed, or errored out, and the item’s callbacks are currently being executed.

The item and the cache should be locked and blocking while an item is in this state.

Complete

The item’s request has completed, or errored out, and the item contains the request’s response text, if any, and any exception thrown.

Expiration date is computed at this transition.

CacheItem Processing

When a WebRequest completes, the matching CacheItem is found, it is marked as Complete and it’s queued callbacks are executed in order.

CacheItem Expiration and Purging

On a poll timer, the items in the cache are checked for expiration and purged if found to be complete and expired. The cache instances should take an internal lock while processing expirations.

NOTE: RequestCache.GetOrCreate also ensures that a qualifying CacheItem is not expired before returning it so a relatively relaxed purge timer, e.g. 10 seconds, can be used on the cache instance.

Cache and Throttle Coupling

In the C# code I have leveraged lambdas/closures to prevent coupling and referencing of the cache and throttle ( a how-not-to lesson learned from soapi).

If this technique presents implementation difficulties in other languages the best place, imo, to maintain references is in the item classes, CacheItem and RequestHolder. Let’s see how the ActionScript and Java implementations shape up to follow up on this.

Share this post


Link to post
Guest sky.sanders

Throttling

Client-side request throttling is necessary for a host of reasons, the most salient being

Ensurance of compliance with Terms of Service (implied or enforced) Drastic simplification of client code with regards to previous concerns

CityIndex.JsonClient.ThrottedRequestQueue follows a pattern initially implemented in Soapi to comply with the Stack Exchange API throttling guidelines which abstracted below:

X number of requests are allowed in any Y timespan (sliding window). Given these parameters further discussion of the real-world use case would include constant rate of request and bursts.

Constant rate of request – A new request may be issued at the interval of Y/X.

Bursts – All X number of requests may be (theoretically) issued in the first millisecond of the sliding window Y timespan but any subsequent requests issued before the completion of Y timespan are in violation of the throttle thresholds.

These modes are conceptual and not exclusive and the throttle implementation should reflect this.

An additional throttle threshold introduced in response to ostensibly out-of-band transport errors is a maximum pending requests limitation. Regardless of compliance with the X per Y thresholds, only Z number of open and active WebRequests are allowed.

The throttle implementation should strive to maintain compliance with TOS and to NOT throw exceptions. Throwing TOS exceptions from the throttle code would require the client code to both watch for TOS exceptions AND throttle itself rendering the inclusion of a throttle in JsonClient redundant.

Any exception in relation to the issuance of the request should be returned in the RequestHolder for processing by the CacheItem callbacks.

Cache/Throttle coupling

As stated in the Caching post, coupling is mitigated by the use of captured references in closures. As the different platforms/languages are implemented this strategy can be examined and re-evaluated

Share this post


Link to post

JsonClient

CityIndex.JsonClient.IJsonClient

 represents the core of the library and exposes 3 methods (with overloads for brevity)

  • BeginRequest<TDTO>
  • EndRequest<TDTO>
  • Request<TDTO>

The first two are obviously an async implementation and the last, 

Request<TDTO>

 (a better name is welcome), presents itself as a sync method but is, internally, a wrapper that calls the async methods.

I chose this route to elimate logic duplication that would be introduced if I were to leverage the sync methods of WebRequest. I made this mistake in Soapi and have never stopped regretting it.

BeginRequest<TDTO>/EndRequest<TDTO>

 present a standard .net async pattern in that you call 

BeginRequest

 with an async callback and the relevant parameters and receive an async result object in the callback.

EndRequest

 is called with the async result object and the result of the request is returned OR any exception encountered during the execution of the request is rethrown. This functionality could be exposed directly on the async result but following the standard .net async pattern allows for a checkpoint in JsonClient for metrics and/or corelation.

...
client.BeginRequest<FooDTO>(MyCallback, state,params);
...     
void MyCallback(ApiAsyncResult<FooDTO> result)
{
      try
   {
      FooDTO response = client.EndRequest(result);
      // do something with response.
   }
   catch(Exception ex)
   {
      // something bad happened
   }
}
Request<TDTO>

, as stated, is a simple blocking wrapper for the async implementation.

...
try
{
   FooDTO response = client.Request<FooDTO>(params);
   // do something with response.
}
catch(Exception ex)
{
   // something bad happened
}

Thus concludes the public surface area of the functionality of the JsonClient library.

While musing about Synchronous vs Asynchronous and the new/alternative Task Parallel Library (TPL)offerings still, I just realized that your BeginRequest()/EndRequest() methods do in fact not fully resemble a standard .net async pattern, in so far there is no IAsyncResult return value available for BeginRequest(), rendering several related usage patterns inapplicable (not the least the TPL’s mentionedTaskFactory.FromAsync wrappers) – is this a deliberate choice?

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  
×