running workflow with custom assembly resolver

Question

Hi,

I have a Workflow stored in a database (as xaml string) that I load at runtime, execute, and persist back to a database. That workflow may reference a set of other dlls that are also stored as byte[] in database as references to that workflow. I tried implementing my own Assembly Resolver by means of retrieving it from database and doing Assembly.Load(byte[]) however for some reason, when I do Load on XAML it loads fine, but when I try to do Run, it complains that certain types are not defined (when my XAML uses types defined in these custom assemblies).

When I put these dlls to an actual executing directory of my process that executes a workflow, it all works fine, as if workflow is not using the results of AppDomain.AssemblyResolve event but instead always tries to look for files in local dir. Which poses a serious issue as I'm trying to build an Azure capable workflow engine and writing to a local bin directory of a certain virtual app is not an option, the only persistency store i have is a database. Plus, even if it won't run on Azure, I would still require to have file write permissions to an executing dir just for the purposes of a single wf execution (which isn't elegant imho)

Thank you

Answer 1

Hi,

Does it throw any error when you try to load custom assembly from database .

If not can you check the "Location" property of the Assembly you loaded.

(Please check whether the assembly you want to load is available in currnet application domain)

What is the value of "Location" property ?

Thanks

---

MB

Answer 2

Oh no, it throws an error, an exception when I try to do Run(). It says it does not know the types in xaml. 'Type abc is not defined' exception message to be precise

Answer 3

My question was does it throw error when you load assembly in Appdomain.

not when you run the workflow.

 

If not can you check the "Location" property of the Assembly you loaded.

(Please check whether the assembly you want to load is available in currnet application domain)

What is the value of "Location" property ?

---

MB

Answer 4

Oh I'm sorry,

no, it doesn't throw an error when I load an Assembly, it loads fine. and Location = "" (probably because I load it from byte[] which is retrieved from db, there is no physical drive location, it's all in memory)

And yes, assembly does belong to AppDomain after I load it:

Assembly loadedAsm = Assembly.Load(wfAsm.BinaryData);
bool tmp = AppDomain.CurrentDomain.GetAssemblies().Any(c => c.FullName == loadedAsm.FullName);

tmp resolved to true.

and by the way, here is an exact error:

The following errors were encountered while processing the workflow tree:
'DynamicActivity': The private implementation of activity '1: DynamicActivity' has the following validation error:   Compiler error(s) encountered processing expression "SampleConfig.Test".
Type 'SampleConfigClass1' is not defined.

 

Answer 5

bump

Answer 6

If you search in the threads you will find a thread of mine with a very similar question.  The short answer is that for now you can't use dynamic assemblies.  The WF4 type resolution classes are coded to explicitly ignore dynamic assemblies.

Answer 7

ok, thank you, though i don't see why would they do that?

writing a file out just for workflow to load and then deleting it from hard drive sounds like a dirty workaround...

Answer 8

If I understand this thread correctly, we had this same problem.   We also save all of our workflows in a database as XAML and save the required custom assemblies in the same database as bytes.   The database also contains a manifest for each workflow that lists the assemblies required by each workflow.   Our rehosted WorkflowDesigner iterates through the Workflow prior to saving the XAML and builds the corresponding manifest. 

We initially tried to have our run-time Worfklow host process load the required assemblies directly from the database service via Assembly.LoadBytes() and had class type resolution problems when loading custom class types from custom assemblies.

We also found that we could run successfully by copying the same assemblies into the Workflow host process's run-time.   That makes the assemblies available by the default Assembly.Load() process (see below), but was not acceptable for a number of reasons.

We resolved this by simply establishing an assembly cache on the machine which is to run the workflow.   We then had the process hosting the workflow pre-load the required custom assemblies via Assembly.LoadFrom() instead of loading it directly from the byte stream via Assembly.LoadBytes().   As odd as it seems, the types resolve correctly when the assemblies are simply loaded by this method rather than loading from bytes.

I tripped to this while searching for class type resolution issues from Loadbytes.   I found a posting on assembly binding contexts in Susan Cook's blog at: http://blogs.msdn.com/b/suzcook/archive/2003/05/29/57143.aspx that made me realize that an assembly is treated differently depending upon the load mechanism.   In that blog entry, she says that there are three different load contexts into which assemblies load.   As odd as it seemed to me when I first read it - and although it does not directly reference the problem with workflows accessing, simply changing the method of loading the assemblies made us completely able to successfully use all manner of custom types with no more type resolution issues.

Here's the relevant section from her Blog.  (You'll want to read the whole thing).   Note the distinction on LoadBytes():

There are three 'contexts' for assembly binding:

• Load context
  In general, if the assembly was found by probing in the GAC, a host assembly store (if hosted), or the ApplicationBase / PrivateBinPaths of the AppDomain, the assembly will be loaded in the Load context.
• LoadFrom context
  In general, if the user provided Fusion a path which was used to find the assembly (and the assembly at that path wouldn't have been found in the Load context), then it's in the LoadFrom context. There are various methods to load by path: LoadFrom(), CreateInstanceFrom(), ExecuteAssembly(), loading an assembly through interop using a codebase, etc.
• Neither
  If the user generated or found the assembly instead of Fusion, it's in neither context. This applies to assemblies loaded by Assembly.Load(byte[]) and Reflection Emit assemblies (that haven't been loaded from disk). Assembly.LoadFile() assemblies are also generally loaded into this context, even though a path is given (because it doesn't go through Fusion).

Hopefully this helps.

-Bob

Answer 9

Hi Bob,

Thank you for your elaborate response, however the runtime I have my workflow cannot make use of a file system. simply put, my application runs on a cloud and only persistence mechanism I have is a database. additionally, file system may get recycled or re imaged at any time (without my knowledge), thus killing the cache you spoke of. having to reconstruct that cache on App start sounds as a bit of an overkill.

Additionally, pre creating that cache is also not an option because my workflow engine lives as a wcf services and there are tools for adding additional assemblies at runtime and tools that actually execute workflows. in my environment workflow creation/assembly referencing/workflow deletion (including having to delete custom assemblies), everything happens at runtime. so in essence, that cache you spoke of would constantly need to be synchronized. And lastly, each workflow execution happens in it's own AppDomain (so that I would be able to clean up loaded assemblies if i need to). I can have 2 different assemblies with the same name (one newer version, one older version) load into two different AppDomains, whereas on file system it would be the same file. And yes, I suppose I could create a cache for each AppDomain I create - but that is slowly starting to turn into synchronization nightmare.

Please excuse me for unmarking your answer as an answer (it is a good and valid answer for different circumstances but unfortunately does not address my case) as I still hope that someone would see this thread in question and will help me with an answer.

If there was a way to use memory stream to load assemblies to various contexts that would be ideal. Or if there was a way to overload Workflows internal assembly resolver, that would also be an option.

Thank you,
Rodion

Answer 10

My gut feeling tells me that Workflow Engine starts a new AppDomain when it starts a new workflow (just the fact that when I execute my workflow, I cannot delete the files it had loaded and after it's finished executing, files can be deleted) tells me that it runs in a different AppDomain which gets unloaded on WF terminate. And because I'm using Load(byte[]) - those assemblies are being loaded to my current AppDomain and not the domain of WF execution.

So at what point does the Workflow Engine creates that AppDomain? is at at Load? or at Run? if At Load, I could then subscribe to AssemblyResolve of that AppDomain and resolve assemblies when I hit Run...

Now if all my observations above are correct and my assumption that AppDomain gets created at Load, then the only thing missing is finding a way to retrieve an AppDomain of that WF execution...

It's a long shot, and please someone tells me if I'm crazy, but if above works, that would make up for a very clean solution...

Rodion

Answer 11

Nope... I was wrong...

public class Program : IExtension
{
	private static void Main(string[] args)
	{

		Console.Out.WriteLine(AppDomain.CurrentDomain.FriendlyName);
		WorkflowApplication wfApp = new WorkflowApplication(new GetDomainActivity());
		wfApp.Extensions.Add(new Program());
		wfApp.Run();
		Console.In.ReadLine();
	}

	public void SetAppDomain(AppDomain domain)
	{
		Console.Out.WriteLine(domain.FriendlyName);
	}
}

public interface IExtension
{
	void SetAppDomain(AppDomain domain);
}

public class GetDomainActivity : CodeActivity
{
	protected override void Execute(CodeActivityContext context)
	{
		IExtension ext = context.GetExtension<IExtension>();
		if(ext != null)
			ext.SetAppDomain(AppDomain.CurrentDomain);
	}
}

Both domains are the same...

that makes absolutely no sense to me how could a workflow unload those assemblies without unloading a domain...

Answer 12

>that makes absolutely no sense to me how could a workflow unload those assemblies without unloading a domain...

Right, but are the assemblies really being unloaded? I would assume they are not, in spite of the file locking behavior you are seeing.

By the way, I think it's great that you unmarked it as answer - when your problem isn't solved, keeping the discussion going is the right thing to do!

Tim

Answer 13

Well, I suppose you're right if shadow copying is used, I haven't really checked assemblies that are actually loaded in AppDomain.CurrentDomain.GetAssemblies().
What I'm doing now is I'm prototyping with AppDomains. I will have each WF execution run in it's own domain, mimic IIS AppPool behaviour...

Though it's not really helping me with an original problem (of not being able to load byte[] assemblies to Load context), it'll address versioning and memory consumption issue.

I've read somewhere on a forum that in order to have WFs load assemblies to no "Load context" I need to rewrite a WF internal assembly resolver. But have not any clues as to how to do that.

If someone knows, I would greatly appreciate if we could elaborate on that topic.

Thank you,
Rodion

Answer 14

As mentioned above there's the 3 assembly contexts 'Load Context', 'LoadFromContext', and 'No Context'?

I believe WF designer would only load things into the Load context by default. If you want something loaded in LoadFrom context or no context, you can load it there yourself, and once you do so, you become responsible for resolving any needed assembly references, via AppDomain event handlers. I'm not sure, but maybe that will answer your question?
Tim

Answer 15

Indeed, calling LoadFrom instead of Load solves the issue. Wierd.

Answer 16

>calling LoadFrom instead of Load solves the issue<

That's the second reason we used a cache directory and why you probably need the approach even if that directory does get wiped out between runs.   You can call it something besides "cache" if you'd like, but you'll still find it much easier to make things work if you copy the assemblies to your local file system and do "LoadFrom" from that location.