How do you manage unmanaged buffers used for P/Invoke?
Background
I started to explore P/Invoke and I decided to create a wrapper for the C strtok function. I chose strtok because I thought that its unusual requirements would be a good start to learning about P/Invoke.
Anyway, here's a short program that uses the wrapper that I would like to create. Notice that the wrapper would be used the same way as strtok is used.
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
class Program
{
public static void Main(string[] args)
{
string a = "abcd-efg,hi///jk|lmnop";
string b = " hello interop";
List<string> tokens = new List<string>();
// get first token of string a using "-" as the delimiter
tokens.Add(Utility.StrTok(a, "-"));
// get next tokens of string a by setting the first argument to null
// each StrTok call can use a different delimiter
tokens.Add(Utility.StrTok(null, ","));
tokens.Add(Utility.StrTok(null, "/"));
// start tokenizing string b without finishing string a
tokens.Add(Utility.StrTok(b, " "));
tokens.Add(Utility.StrTok(null, " "));
tokens.Add(Utility.StrTok(null, " "));
foreach (string token in tokens) {
Console.WriteLine("<{0}>", token);
}
}
}
/* Output
<abcd>
<efg>
<hi>
<hello>
<interop>
<>
*/
Here's the wrapper for strtok.
class Utility
{
[DllImport("msvcrt.dll", CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
static extern IntPtr strtok(IntPtr str, string delim);
static IntPtr strtokBuffer = IntPtr.Zero;
public static string StrTok(string str, string delim)
{
// if str is null
// continue tokenizing the same unmanaged string from previous strtok call
// if str is not null
// strtok will start tokenizing a new unmanaged string initialized with str
// if the token returned from strtok is IntPtr.Zero
// strtok has finished tokenizing the unmanaged string
IntPtr token;
if (str == null) {
token = strtok(IntPtr.Zero, delim);
} else {
IntPtr strAnsi = Marshal.StringToHGlobalAnsi(str);
token = strtok(strAnsi, delim);
Marshal.FreeHGlobal(strtokBuffer);
strtokBuffer = strAnsi;
}
if (token == IntPtr.Zero) {
Marshal.FreeHGlobal(strtokBuffer);
strtokBuffer = IntPtr.Zero;
return null;
}
return Marshal.PtrToStringAnsi(token);
}
}
Problem
The problem is that strtok remembers its position in the string used in the previous call to strtok. Because the string used by strtok is unmanaged, I have to make sure that the unmanaged string is available until strtok finishes tokenizing the entire string,
or until strtok receives a new string to tokenize. However, that means that if strtok doesn't finish tokenizing the string, or if strtok doesn't get a new string to tokenize, then the unmanaged string will never be freed.
So, how should I (writer of the wrapper, or user of the wrapper, or both) manage the lifetime of the unmanaged string used by strtok?
Note:
I tried importing strtok as "static extern string strtok(StringBuilder str, string delim)" so I wouldn't have to deal with the unmanaged string. When I used it in a small program, the program compiled and returned the expected results in
one computer, but when I compiled and ran the same program in another computer, the program crashed. Ideally, I would like to make the wrapper make it seem like that the import actually worked.
