Remembering the Future

The one solution that may solve this, that I haven't tried, is to use a custom thread pool using reflection to call private .NET framework methods. Mike Woodring has an example of this.

Other solutions involve:
Invoke style asynch function/WebService calls with an empty End handler:
This holds on to the page/thread depleting the ASP.NET Thread pool

Calling ThreadPool.QueueUserWorkItem in a synch Web Service
This also steals threads and uses the IsBackground property set to true. "When you do this, you're saying, "It's okay if the process shuts down while this thread is still running."

using the OneWay=true attribute of WebServices:
There is a bug that drops the thread context allowing things like integrated security (no sql or file handling)

Currently I'm just calling a sync WebService that starts its own thread and immediately returns, buffering the database connection and filestreams in the thread class before starting the thread. This also leaving me to manage my own threads.

This seems way to hard, for something that should be extremely simple.

Topics: Async | Threads

Traditional tools require the data schema to be developed prior to the creation of the data. Unfortunately, sometimes the data schema emerges only after the software is already in use—and the schema often changes as the information grows. A typical example is the information contained in the item descriptions on eBay. It seems impossible for the eBay developers to define an a priori schema for the information contained in such descriptions. Today, all of this information is stored in raw text and searched using only keywords, significantly limiting its usability. The problem is that the content of item descriptions is known only after new item descriptions are entered into the eBay database. EBay has some standard entities (e.g., buyer, date, ask, bid...), but the meat of the information—the item descriptions—has a rich and evolving structure that isn’t captured.

Traditional software design methodology does not work in such cases. One cannot rigidly follow the steps:

1. Gather knowledge about the data to be manipulated by the software components being designed.
2. Design a schema to model this information.
3. Populate the schema with data.

We need software and methodologies that allow a more flexible process in which the steps are interleaved freely, while at the same time allowing us to process this information automatically.

Topics: XML | RDF | Schema

BigTable is a system for storing and managing very large amounts of structured data. Data is organized into tables with rows and columns, but unlike a traditional database system, the row/column space can be sparse. Row keys and values are arbitrary strings, and the system allows each row/column cell to store not just a single value but a set of values with associated timestamps, simplifying analyses that examine how values have changed over time. Data in a single table is internally broken at arbitrary row boundaries to form contiguous regions of data called tablets. These tablets are distributed across a large pool of worker machines. The system is designed to manage several petabytes of data distributed across thousands of machines, with very high update and read request rates coming from thousands of simultaneous clients.

Topics: Google | BigTable

Time is the concept that results when consciousness sees things changing state. Events are what happens when things change. Senses are the things we humans use to witness those events.

. . .

When I blog, I metamorphose myself for brief moments into an event stream source. I change things. Changing things triggers events. Other people and other processes, peruse the event stream I create. I feed the leeches.

I do this with a 'feed'. A periodically emitted atom [1] of change. Something to react to. Something to sense. Something to indicate the very passage of time.

. . .

What if we drag the red rose onto the web? What would that involve? Well, in the digital world, we work with representations of things, rather than smelly, oily visually arresting reality. Let us give our rose a URL:

www.example.com/rose

. . .

Think about what you do all day. How much of it is down to responding to change? Quite a proportion I would imagine. How much business activity revolves around how things change with respect to time? Prices, stock levels, revenues, manufacturing rates, these are all examples of things that change with respect to time. Things for which monitoring any changes is key to the way we work.

Technologies like Atom and RSS, originally heralded as tools to allow diary-oriented web-pages (weblogs) and diary-oriented feeds to be mass produced, are something much grander to my mind.

They are nothing less than the addition of a new dimension to the web.

The dimension of time.

Topics: RSS | Time | Workflow