MSDOTnet.org Forum Index
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

interrupting flow of a function and/or yielding control

 
Post new topic   Reply to topic    MSDOTnet.org Forum Index -> Common Language Runtime (CLR)
Author Message
dB.



Joined: 21 Dec 2007
Posts: 1

PostPosted: Fri Dec 21, 2007 5:41 pm    Post subject: interrupting flow of a function and/or yielding control Reply with quote

I am trying to build a workflow system for database detection that
needs to perform thousands of detections in parallel. Most of the time
the detectors sit waiting on network IO to do something. The actual
detector code is fairly thick, with a third party implementing the
actual detectors.

What I am looking to provide is an engine in which I can create a list
of 1000 detectors, then execute the code in each detector all at the
same time without spawning 1000 threads.

Most of the delay is on the network. If I were writing the detectors
myself, I could do something like this:

BeginDetection()
{
...
send a packet
begin receive a packet (callback on OnReceivePacket)
}

OnReceivePacket(...)
{
... finish detection
}

Rebuilding all detectors this way is cumbersome for our purpose.

The question is: can CLR do something for me in terms of interrupting
the flow of a function, saving the stack and coming back to it? The
detectors could yield control too in the appropriate places
explicitly.

Thx
dB.

Archived from group: microsoft>public>dotnet>framework>clr
Back to top
View user's profile Send private message
Jeroen Mostert



Joined: 20 Dec 2007
Posts: 18

PostPosted: Sat Dec 22, 2007 5:35 am    Post subject: Re: interrupting flow of a function and/or yielding control Reply with quote

dB. wrote:

> What I am looking to provide is an engine in which I can create a list
> of 1000 detectors, then execute the code in each detector all at the
> same time without spawning 1000 threads.
>
Well, you obviously can't literally do that -- if you want to execute the
code "all at the same time" then multiple threads are inevitable, otherwise
the code can at best be "not quite at the same time". However, I assume that
you just mean that you'd like to limit the number of active threads by not
dedicating them to waiting on I/O.

> Most of the delay is on the network. If I were writing the detectors
> myself, I could do something like this:
>
> BeginDetection()
> {
> ...
> send a packet
> begin receive a packet (callback on OnReceivePacket)
> }
>
> OnReceivePacket(...)
> {
> ... finish detection
> }
>
Indeed, that's the classic implementation of asynchronous requests, which
will use a very efficient mix of thread pooling and completion ports in .NET.

> Rebuilding all detectors this way is cumbersome for our purpose.
>
I nevertheless strongly suggest that you consider it. A rewrite to move to
asnychronous I/O, while costly, is something you only do once (well, for
every codebase) and it continues to pay off. Although it may be
"cumbersome", for the most part it's not hard.

> The question is: can CLR do something for me in terms of interrupting
> the flow of a function, saving the stack and coming back to it? The
> detectors could yield control too in the appropriate places
> explicitly.
>
What you're asking for is a coroutine, something which is not natively
implemented by the framework. That said, you can implement this using the
unmanaged hosting interfaces (that means leaving the comfortable world of
..NET and entering the harsh environs of C++ and Win32). The CLR associates
managed "tasks" with OS threads, and the CLR host can control this
assignment. Take a look at the IHostTaskManager interface and the ICLRTask
interface, and especially the SwitchIn() and SwitchOut() methods of the
latter. Using these, I suspect you could build a coroutine implementation
fairly straightforwardly, possibly using Win32 fibers to ease some of the
load (though there are many, many "details" to get right).

Even so, what you want is not quite comparable to a pure coroutine scenario.
Even if your detector can yield explicitly, it cannot do so *during* I/O
(because that code is not under your control), so it could at best yield
*between* I/O requests. But if I/O is what you're mostly doing, this is of
little use. Your threads will still be preoccupied with idling on I/O.
Managing threads explicitly will allow you to cut down on the number of
threads, but if those few threads are mostly busy doing nothing you haven't
gained much in terms of scalability. Even an unmanaged host has no way of
detecting when code is "waiting on I/O" to reliably switch out the task.

You can detect when the task is waiting in general, though. Implementing the
IHostSyncManager interface will give you precise control over the
synchronization primitives used by the managed code, and most synchronous
I/O is implemented by eventually using one of these primitives to wait for
completion. However, leveraging this effectively to turn threads into
coroutines without introducing deadlocks is a daunting task, to say the
least. Multithreaded programming is difficult enough without having to worry
about the implementation of the synchronization primitives themselves.

If the above sounds complicated to you, that's because it is. If you really
want to go this route, then pick up Steven Pratschner's "Customizing the
Microsoft .NET Common Language Runtime" (ISBN 9780735619883). This book is
pretty much not optional if you want to sink your teeth in hosting, because
the documentation, while pretty good, does not give you the big picture, let
alone the many pitfalls. It took me a good deal of two weeks to implement a
pretty simple host that uses AppDomains as lightweight, reliable,
restartable processes, and that doesn't even touch the more difficult
aspects of hosting. Something as dramatic as what you're asking for sounds
like a multi-month project for the uninitiated, and that's assuming you're
willing/able to muck about with unmanaged code in the first place.

If all you're concerned about is the amount of code you'll have to write to
make the detectors use asynchronous I/O, then you're probably still better
off hacking together some sort of code generator/translator that will
convert the synchronous calls to asynchronous ones for you, or possibly
doing even more dramatic rewrites of the code. While not a pretty solution,
it's still much less involved than implementing coroutines at a low level.

--
J.
Back to top
View user's profile Send private message
Dave Farquharson



Joined: 19 Nov 2007
Posts: 4

PostPosted: Fri Dec 21, 2007 9:08 pm    Post subject: Re: interrupting flow of a function and/or yielding control Reply with quote

This is a pretty awesome and well thought out reply, and as someone who just
implemented a CLR hosting component myself I also heartily recommend Steven
Pratschner's book if you're thinking about using CLR hosting. It would have
taken me 4 times as long to get working without that book.

-dave


"Jeroen Mostert" wrote in message $0$85783$e4fe514c@news.xs4all.nl...
> dB. wrote:
>
>> What I am looking to provide is an engine in which I can create a list
>> of 1000 detectors, then execute the code in each detector all at the
>> same time without spawning 1000 threads.
>>
> Well, you obviously can't literally do that -- if you want to execute the
> code "all at the same time" then multiple threads are inevitable,
> otherwise the code can at best be "not quite at the same time". However, I
> assume that you just mean that you'd like to limit the number of active
> threads by not dedicating them to waiting on I/O.
>
>> Most of the delay is on the network. If I were writing the detectors
>> myself, I could do something like this:
>>
>> BeginDetection()
>> {
>> ...
>> send a packet
>> begin receive a packet (callback on OnReceivePacket)
>> }
>>
>> OnReceivePacket(...)
>> {
>> ... finish detection
>> }
>>
> Indeed, that's the classic implementation of asynchronous requests, which
> will use a very efficient mix of thread pooling and completion ports in
> .NET.
>
>> Rebuilding all detectors this way is cumbersome for our purpose.
>>
> I nevertheless strongly suggest that you consider it. A rewrite to move to
> asnychronous I/O, while costly, is something you only do once (well, for
> every codebase) and it continues to pay off. Although it may be
> "cumbersome", for the most part it's not hard.
>
>> The question is: can CLR do something for me in terms of interrupting
>> the flow of a function, saving the stack and coming back to it? The
>> detectors could yield control too in the appropriate places
>> explicitly.
>>
> What you're asking for is a coroutine, something which is not natively
> implemented by the framework. That said, you can implement this using the
> unmanaged hosting interfaces (that means leaving the comfortable world of
> .NET and entering the harsh environs of C++ and Win32). The CLR associates
> managed "tasks" with OS threads, and the CLR host can control this
> assignment. Take a look at the IHostTaskManager interface and the ICLRTask
> interface, and especially the SwitchIn() and SwitchOut() methods of the
> latter. Using these, I suspect you could build a coroutine implementation
> fairly straightforwardly, possibly using Win32 fibers to ease some of the
> load (though there are many, many "details" to get right).
>
> Even so, what you want is not quite comparable to a pure coroutine
> scenario. Even if your detector can yield explicitly, it cannot do so
> *during* I/O (because that code is not under your control), so it could at
> best yield *between* I/O requests. But if I/O is what you're mostly doing,
> this is of little use. Your threads will still be preoccupied with idling
> on I/O. Managing threads explicitly will allow you to cut down on the
> number of threads, but if those few threads are mostly busy doing nothing
> you haven't gained much in terms of scalability. Even an unmanaged host
> has no way of detecting when code is "waiting on I/O" to reliably switch
> out the task.
>
> You can detect when the task is waiting in general, though. Implementing
> the IHostSyncManager interface will give you precise control over the
> synchronization primitives used by the managed code, and most synchronous
> I/O is implemented by eventually using one of these primitives to wait for
> completion. However, leveraging this effectively to turn threads into
> coroutines without introducing deadlocks is a daunting task, to say the
> least. Multithreaded programming is difficult enough without having to
> worry about the implementation of the synchronization primitives
> themselves.
>
> If the above sounds complicated to you, that's because it is. If you
> really want to go this route, then pick up Steven Pratschner's
> "Customizing the Microsoft .NET Common Language Runtime" (ISBN
> 9780735619883). This book is pretty much not optional if you want to sink
> your teeth in hosting, because the documentation, while pretty good, does
> not give you the big picture, let alone the many pitfalls. It took me a
> good deal of two weeks to implement a pretty simple host that uses
> AppDomains as lightweight, reliable, restartable processes, and that
> doesn't even touch the more difficult aspects of hosting. Something as
> dramatic as what you're asking for sounds like a multi-month project for
> the uninitiated, and that's assuming you're willing/able to muck about
> with unmanaged code in the first place.
>
> If all you're concerned about is the amount of code you'll have to write
> to make the detectors use asynchronous I/O, then you're probably still
> better off hacking together some sort of code generator/translator that
> will convert the synchronous calls to asynchronous ones for you, or
> possibly doing even more dramatic rewrites of the code. While not a pretty
> solution, it's still much less involved than implementing coroutines at a
> low level.
>
> --
> J.
Back to top
View user's profile Send private message
Ben Voigt [C++ MVP]



Joined: 08 Aug 2007
Posts: 189

PostPosted: Mon Dec 24, 2007 6:33 pm    Post subject: Re: interrupting flow of a function and/or yielding control Reply with quote

"Jeroen Mostert" wrote in message $0$85783$e4fe514c@news.xs4all.nl...
> dB. wrote:
>
>> What I am looking to provide is an engine in which I can create a list
>> of 1000 detectors, then execute the code in each detector all at the
>> same time without spawning 1000 threads.
>>
> Well, you obviously can't literally do that -- if you want to execute the
> code "all at the same time" then multiple threads are inevitable,
> otherwise the code can at best be "not quite at the same time". However, I
> assume that you just mean that you'd like to limit the number of active
> threads by not dedicating them to waiting on I/O.
>
>> Most of the delay is on the network. If I were writing the detectors
>> myself, I could do something like this:
>>
>> BeginDetection()
>> {
>> ...
>> send a packet
>> begin receive a packet (callback on OnReceivePacket)
>> }
>>
>> OnReceivePacket(...)
>> {
>> ... finish detection
>> }
>>
> Indeed, that's the classic implementation of asynchronous requests, which
> will use a very efficient mix of thread pooling and completion ports in
> .NET.
>
>> Rebuilding all detectors this way is cumbersome for our purpose.
>>
> I nevertheless strongly suggest that you consider it. A rewrite to move to
> asnychronous I/O, while costly, is something you only do once (well, for
> every codebase) and it continues to pay off. Although it may be
> "cumbersome", for the most part it's not hard.
>
>> The question is: can CLR do something for me in terms of interrupting
>> the flow of a function, saving the stack and coming back to it? The
>> detectors could yield control too in the appropriate places
>> explicitly.
>>
> What you're asking for is a coroutine, something which is not natively
> implemented by the framework. That said, you can implement this using the

Actually, the C# yield return statement implements coroutines.

You're correct that it doesn't help with I/O particularly.... unless....
oooh I have an idea.

Make an IEnumerable interface that yield returns some I/O descriptor, which
a main loop will start asynchronously, providing an appropriate callback.
The callback will invoke IEnumerable.GetNext() on the associated detector,
starting the next I/O asynchronously.
Back to top
View user's profile Send private message
Jeroen Mostert



Joined: 20 Dec 2007
Posts: 18

PostPosted: Tue Dec 25, 2007 4:58 am    Post subject: Re: interrupting flow of a function and/or yielding control Reply with quote

Ben Voigt [C++ MVP] wrote:
> "Jeroen Mostert" wrote in message
> $0$85783$e4fe514c@news.xs4all.nl...

>> What you're asking for is a coroutine, something which is not natively
>> implemented by the framework. That said, you can implement this using the
>
> Actually, the C# yield return statement implements coroutines.
>
Well, sort of. It's not intended as a general coroutine construct, since the
routine ends aren't equals (GetNext() decides which iterator to call, but
the iterator doesn't decide which routine to yield control to). "Iterators
with closures" is more like it. But yeah, with a prearranged control type
for a return value, you could probably implement every coroutine scenario.
If you don't mind some pretty unintuitive code.

> You're correct that it doesn't help with I/O particularly.... unless....
> oooh I have an idea.
>
> Make an IEnumerable interface that yield returns some I/O descriptor, which
> a main loop will start asynchronously, providing an appropriate callback.
> The callback will invoke IEnumerable.GetNext() on the associated detector,
> starting the next I/O asynchronously.
>
This sounds like much more work than the OP was gunning for. (Of course, my
hosting suggestion is even *more* work, but hey.)

There are still issues with scalability in this approach, only now you've
shifted the threading issues to the thread pool (assuming this is what we'll
use to kick off the asynchronous requests). The main problem here is that,
whichever way you slice it, the I/O will be done synchronously, so it'll tie
up a thread. If you care about the number of threads involved, you just
can't have 1,000 detectors going simultaneously, because it's going to take
1,000 threads. The only real solution is to rewrite the I/O to be asynchronous.

Using the thread pool does have the benefit of not blowing up the system,
since there's a limit to the number of requests it can have in flight. If
you *must* do lots of synchronous things with as much parallelism as
possible, the thread pool is probably the best way to go.

Actually, the detectors *might* just be doing lots of small I/O requests
that could profitably be broken up with a coroutine pattern, but the OP
hasn't really made it clear whether this is the case. Nor is it clear how
much rewriting of the detector code is acceptable, really.

--
J.

Back to top
View user's profile Send private message
Display posts from previous:   
Related Topics:
Function like strtok - clr Does anybody know of a function under the clr that is like Cs strtok or am I going to have to accomplish the same thing?

Compile users string into function module? Hello all, Is there a way to take a user entered string and Compile it into a module or function that I can use from within my program. I want to allow the users to enter C# code that will fill a function that I could call from within my code. I would lik

possible to call javascript from IE hosted winform control ? Hello everyone, Can anybody tell me if it is possible to call a javascript function in the html from a embedded winform control ? Note. this is not ASP, just a plain html page with a javascript function such as function Hello(name) { alart("Hello "+name);

System.Windows.Forms.Control based classes are not collected Hello, I'm not able to figure out how to collect/ dispose classes based on Under Windows CE the memory will fill up until i get an out of memory exception. On my PC everything works fine. I'm running Windows CE 5. What can I

Build your own Visio 12 Like Flow/Diagram App with C#,VB, De UCCDraw Component V10.20 Released! November 30, 2006 For more information (press only) please contact: Paul Chi UCanCode Software, Inc. Phone: (86) 28-851065
Post new topic   Reply to topic    MSDOTnet.org Forum Index -> Common Language Runtime (CLR) All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group