locked
Are the feature sets locked?

    Question

  • I'm just curious, is the feature set for C# 4.0 locked? There is one thing that I feel is a glaring omission, and really hurts C#. The fact that generics do not support operator constraints. This is a huge issue for many of us. Why can't we do:

    public class List<K> where operator+ 

    I know this would make my life easier, I guess I'm just wondering if it's definitely been tabled or if it's still a possibility for 4.0
    Wednesday, November 05, 2008 3:12 PM

All replies

  • Heck, why stop with operator constraints?

    Why not introduce a concept of a "signature" (like an interface) for structural typing and constraints?

    (You might recognize this as a partial definition of an algebraic Ring.)

    public signature Numeric<T>
    {
        static T operator+(T left, T right);
        static T operator*(T left, T right);
        static T Zero { get; }
        static T One { get; }
        // ...
    }

    A signature would be used to declare a structural typing relationship in contrast with the nominative typing expressed by interface implementation and class inheritance.  You might also think of this as a variation of "duck typing".

    http://c2.com/cgi/wiki/Wiki?NominativeAndStructuralTyping

    Signatures could then be used to create generic type constraints like the one you illustrated:

    public static class Math
    {
        public static T Sum(IEnumerable<T> values)
            where T : Numeric<T>  // signifying conformance to a signature, NOT inheritance
        {
            T total = T.Zero;
            foreach (T value in values)
                total += value;  // valid because we know T must have an operator+
            return total;
        }
    }

    This sort of constraint is somewhat more difficult to implement than the standard ones currently offered by the CLR.  That's because the JIT would have to generate different bodies for different types that conform to the same signature since they can have completely unrelated internal structure (think v-tables).  Currently the JIT only has to generate separate bodies for generic specializations involving value types.

    Incidentally, the situation gets immediately more complex for primitive types like Int32.  You might have noticed that Int32 doesn't even have an operator+ defined on it.  There's a different bytecode altogether used to express integer addition in the IL.

    So all told... this would be a lot of work.  But I do feel it would be very useful in the long term.

    There was a C++ proposal of this idea a while back.  http://docs.freebsd.org/info/gcc/gcc.info.C++_Signatures.html

    Of course... there's always Haskell...

    Thursday, November 06, 2008 7:28 AM
  •  DMeglio, Jeffrey,

    The feature sets are not locked, and changes are still possible, but I want to make sure that you understand that it is unlikely that we will add new features at this time. Not impossible, but unlikely.

    We are, however, very interested in comments on the existing features outlined for C# 4.0. Please tell us if you think the features in the CTP are interesting, and if we are implementing them correctly. We are no longer early in the cycle, but we are not completely locked down, and we are serious about taking your feedback; change is not impossible.

    I'll try to get one of the language PM's to respond to your question about operator constraints. This request, and other requests about constraints come up quite often, and we have given several of them serious thought. I don't, however, want to get your hopes up about it, but I do want to assure you that we both hear your current feedback, and have looked at this one carefully before. Thanks Jeffrey, for your suggestion, also. Have you watched Anders' talks from PDC? His comments on the new Dynamic object and some of our future plans for the compiler touch on the subject of "duck typing." I think particularly in the repeat of his talk on The Future of C#. See the front page of the Dev Center for links to at least the first version of that talk.

    - Charlie Calvert 
    CSharp Community Program Manager
    Thursday, November 06, 2008 7:43 AM
    Moderator
  • I think something like this signature proposal is orders of magnitude more important than the "dynamic" feature. Calling a dynamic method can be made reasonably simple without extending the language. But not being able to do calculations on generic types is a fundamental problem of C# that forces many people to keep using C++. 

    The signature proposal would also solve another problem: there are a lot of features in the C# language which are not accessible when using generics. Static methods, constructors with arguments, etc. The signature proposal would solve this problem just fine.

    I noticed this problem with generics four years ago and created a feedback item: https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=94264

    Many people seem to agree with me that this is very important, but microsoft does not do anything about it. Instead we get a dynamic keyword. 

    There are a lot of other big inconsistencies and problems with the C# language, corlib and the .NET language. Such as this problem with System.Enum: Is there any reason to not let an enum implement IEquatable<T>?
    https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=374647

    And the very big struct inlining fiasco is still not solved for the x64 architecture:
    https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=93858

    The only new C# 4.0 language feature I like is covariance and contravariance. But I do not see why it is limited to interfaces and delegates. 

    I am ashamed to say that I actually recommended C# to friends of mine. I am still using it for all my current projects, but for a new very large project I am considering moving to Scala.
    Thursday, November 06, 2008 8:45 AM
  • Thanks Charlie.  :-)
    I actually attended Anders' talk at the PDC last week.  Great stuff!

    Think of signatures as a statically typed (and checked!) variation on what's possible using the new C# dynamic typing features.  This behavior is often desirable since the static type information itself is a form of documentation that can flow through the system.

    That said, I would REALLY love to see some kind of "monadic" function composition scope.  Ok, I know "monad" is a "bad word" so let me use a different name.  How about "workflows" like in F#?

    A workflow would permit a limited form of extensible syntax based on lambdas taking the appearance of blocks?  For example, I'd like to be able to write the following kinds of things:

    Assert x == 4;
    Assert x == 4, y == 42;
    Parallel x.DoThis(), y.DoThat(), { z.DoSomething(); z.DoSomethingElse() };
    Future<int> future = Future x.DoSomething();

    The idea is that the statement would be evaluated by creating a new instance of some Workflow (monad) class bound to a delegate or expression tree for each expression within the block, combining them together, then finally invoking the Evaluate method to produce a result (possibly void).

    You can think of these operations as monad Bind, Plus and Return if you like.

    Here's a possible implementation of an Assert workflow.

    public class AssertWorkflow : Workflow
    {
        private readonly List<Expression<Func<bool>>> conditions;

        private AssertWorkflow(List<Expression<Func<bool>>> conditions)
        {
            this.conditions = conditions;
        }

        // think of this as Monad-Bind.
        public AssertWorkflow(Expression<Func<bool>> condition)
            : this(new List<Expression<Func<bool>>> { condition }))
        {
        }

        // think of this as Monad-Plus.
        public AssertWorkflow Combine(AssertWorkflow right)
        {
            List<Func<bool>> conditions = new List<Expression<Func<bool>>>(conditions);
            conditions.AddAll(right.conditions);
            return new AssertWorkflow(conditions);
        }

        // think of this as Monad-Return.
        public void Evaluate()
        {
            foreach (Expression<Func<bool>> condition in conditions)
                if (! condition.Invoke())
                    throw new AssertionFailedException(condition.ToString());
        }
    }

    The compiler would actually emit the following code for the Assert example seen above (note that I did not expand the expression trees outright to keep things clear):

    new AssertWorkflow(() => x == 4).Combine(new AssertWorkflow(() => y == 42).Evaluate();

    This is all just compiler magic and requires no changes to the CLR.  However the impact on C# is quite significant.  You can now define convenient new syntax for all sorts of stuff.

    With a little imagination, we can extend the syntax above with some other unambiguous extensions like infix workflows that behave a bit like operators.

    eg. Assert list.Count == 4 Because "We added 4 elements to the collection";

    public class BecauseWorkflow : Workflow
    {
        public Because(string reason)
        {
            Reason = reason;
        }

        public string Reason { get; private set; }

        // no Combine or Evaluate method so this workflow cannot appear on its own...
    }

    public class AssertWorkflow : Workflow
    {
        private readonly List<Expression<Func<bool>>> conditions;
        private readonly string reason;

        private AssertWorkflow(List<Expression<Func<bool>>> conditions, string reason)
        {
            this.conditions = conditions;
            this.reason = reason;
        }

        public AssertWorkflow(Expression<Func<bool>> condition)
            : this(new List<Expression<Func<bool>>> { condition }))
        {
        }

        public AssertWorkflow Combine(AssertWorkflow right)
        {
            List<Func<bool>> conditions = new List<Expression<Func<bool>>>(conditions);
            conditions.AddAll(right.conditions);
            return new AssertWorkflow(conditions, right.reason);
        }

        public AssertWorkflow Combine(BecauseWorkflow right)
        {
            return new AssertWorkflow(condition, right.Reason);
        }

        public void Evaluate()
        {
            foreach (Expression<Func<bool>> condition in conditions)
                if (! condition.Invoke())
                    throw new AssertionFailedException(reason, condition.ToString());
        }
    }

    I'm sure this idea can be improved further.  For example, I haven't talked about anything analoguous to a monad transformer.   Neither have I discussed how workflows could be further composed or how a workflow might introduce new symbols or binding in lexical scope.  However, even the simple notation described above can be extraordinarily useful.

    Jeff.

    P.S.  I've long felt that syntax extensions like LINQ should not themselves be first class.  Instead they should be expressible using some other first-class notation that can capture the necessary semantics.

    P.P.S.  Please do pass these ideas along.  I'd love to start a discussion about some relatively straightforward ways we could make the C# and VB languages more expressible.

    P.P.P.S.  Another favourite idea of mine is that of expanding the C# metaprogramming features to enable compile-time features similar to Python "decorators" or LISP hygenic macros.


    Thursday, November 06, 2008 9:57 AM
  • Charlie,

    I appreciate the reply. If I had to be completely honest, I'm very disappointed in what I see coming in C# 4.0. Optional parameters is nice, I've missed that from C++.

    Dynamic typing... Gotta say, I hate it. I've worked in ActionScript and JavaScript on many occassions and dynamic typing has been the bane of my existence. I don't know how many times I've done obj.ToString() in ActionScript and it compiles just fine. Then I run the code and BOOM. Why? Well I got my casing screwed up, I was thinking C#. In ActionScript it's obj.toString(). I don't intend to use the dynamic typing system at all, I see no positives of it for my line of programming.

    Contravariance and covariance support... nice start, but it sounds like you're only scratching the surface with that one. It only works with reference conversion? Why? Going from IEnumerable<int> to IEnumerable<object> to me, is important, and this implementation does not allow it.

    All the little COM hacks I hate, auto passing by ref? That's not a disaster waiting to happen! And what happens when the call actually DOES return something (you know, what pass by ref is all about?) Thanks to MS, the value just gets lost in the ether instead of me getting a compile time error. To promote programmer laziness, you're introducing a whole new set of bugs. From what I gather (and I'm by no means a COM expert) the COM community would much rather see default and indexed properties rather than auto pass-by-ref.

    Like I said, overall,  I see nothing in C# 4.0 that I'm super-excited about. 2.0 brought me generics, it was a huge thing, 3.5 brough us LINQ, another leap forward, 4.0, for me, doesn't offer anything. With the feature set as it is currently defined, I'm not sure it will be worth my time/effort to upgrade.

    Instead of adding new features that really just appeal to niche groups, I think it's time for C# to take care of some of the glaring omissions. In my mind, the ability to use operators is one of those glaring omissions. I'd love to see something like Jeffrey mentioned with signatures, but at the very least, SOMETHING needs to be done.
    Thursday, November 06, 2008 4:08 PM
  • DMeglio,
    Actually I like some of the new dynamic typing features in C#.  Certainly the new "dynamic" static type is better than the earlier proposal of scoped dynamic regions.

    That said, I too dislike the amount of effort being invested in COM-specific features.  All this "No PIA" stuff and optional "refs".  Blech.  Unmanaged interfaces belong to a different world than managed ones.  They adhere to very different usage conventions.  At least some of that different should be expected to leak through the API.  A sensible app developer will then wrap these interfaces behind a facade that feals with the specific interop concerns locally.  Still... I'm sure it'll help someone out there.

    Anyway, I'd prefer to see more regularity in the language overall.

    The issue with generic variance and value types is right on the money, IMHO.  But I do understand that it's significantly harder to implement efficiently.

    If we relax the requirement of maintaining object identity during a variance conversion then we can treat a conversion to/from a generic interface specialization involving value types as something similar to a boxing operation.

    For example, suppose we try to convert an IEnumerable<int> to IEnumerable<object>.  The CLR could create a box of type IEnumerable<object> containing the original IEnumerable<int>.  The box's v-table would refer to a set of generated wrapper methods to adapt the interface appropriately.  The generated IEnumerable<object>.get_Current method would work by wrapping the call to IEnumerable<int>.get_Current with a box operation on the result.

    A similar situation would occur when converting IComparer<object> to IComparer<int>.  Here the generated IComparer<int>.Compare method would work by wrapping the call to IComparer<object>.Compare with a box operation on both parameters.

    Of course, this strategy will only work for generic interfaces (and potentially generic delegates via thunking) but it's a step in the right direction.  It's not a perfect solution but it does handle most interesting cases.

    Jeff.
    Thursday, November 06, 2008 8:40 PM
  • DMeglio,
    I wrote a little more about the Signatures idea on my blog.  I'll probably also post some bits about workflows.  It's all a little rough though.

    http://blog.bits-in-motion.com/2008/11/c-signatures.html

    Cheers,
    Jeff.
    Friday, November 07, 2008 3:57 AM
  • I like what you've said about structural typing, but I'm not sure I buy the extension part. Mainly this is because I'm not a huge fan of extension methods to begin with! It seems like dynamic typing is only going to make extension methods worse by preventing the two technologies from playing nicely together, at least for C# 4.0. It's really more of a personal opinion than it is a logical argument against it! In any case, your structural typing argument sounds very much like something already in C# (since 2.0 I believe)... delegate inference. I don't have to do new MyDelegate. As long as my delegate matches the signature, C# "infers" that it is the same. The implementation is of course very difference, but it does show that C# already has some of this structural relationship logic going on. I think having more of it would be great.

    One of the things that has bothered me most about C#  is all the little caveats. Yes, I know every language has its caveats, but it seems like C# just adds more and more and never fixes any. What I mean by this is things like Jeffrey said, why can I have an extension method, but no extension property? A property is essentially just a get_PropName and set_PropName when it gets down to the CLR level, isn't it? So I can create extension get_PropName() and set_PropName(value), but I can't create the abstraction of PropName. Automatic properties added in 3.5 were a nice time saver, but there is no way to initialize them outside of setting it in the constructor. Why can't I initialize it in the declaration? I could go on...

    My fear is that C# 4.0 is adding more of these caveats instead of fixing them. The variance changes are a step in the right direction, but not perfect. Dynamic types adds a new caveat of not working with extension methods. Pretty much all of the new C# stuff with the exception of optional parameters is adding new limitations on what we can do. I know it'll be many years before C# is a "perfect" language, but in my opinion, each version should make strides toward removing some of the pre-existing limitations. 2.0 removed many headaches, 3.0, even more. 4.0 though it gives us some new features, seems like it is going to create more headaches than it will relieve.

    While the limitation of not being able to use generics with operators is huge to me, I doubt it's the most important issue in C#. But at the same time, it seems like something that could be addressed and would make the language more robust. I hate that I could use operators in C++ templates 10 years ago but I still can't use them today in C# generics (yes, I understand why, C++ templates are completely different, but to the end user they accomplish the same goal and one has a limitation that the other does not).
    Sunday, November 09, 2008 6:44 PM
  • Re using operators and generics; I have a .NET 3.5 answer for that in MiscUtil; free to use and available now:


    Re C# 4.0, the langauge doc suggests that operators will be supported on "dynamic", but it isn't implemented yet so I can't properly assess it:


    Marc [C# MVP]
    Tuesday, November 11, 2008 10:12 AM
  • Re: Marc Gravell

    The solution proposed by you still has the overhead of a delegate invocation, if I understood it correctly. So there is a significant overhead. If that overhead is acceptable depends on what you want from C#. 

    If you use C# for scripting office and for writing in-house business applications, then this is of course acceptable.

    If you want to use C# as a general purpose programming language that can be a successor to C++ for writing high-performance applications, then this is totally unacceptable.

    But I guess microsoft has decided that managed code will never be fast enough for serious work, so they just don't care anymore. For example new windows 7 apis like Direct2D are being released as unmanaged APIs. There may or may not be a managed wrapper in the future, but the fact remains that high performance APIs are not even exposed to the managed world anymore. 

    The C# team seems to concur with this. Most new features seem to be targeted to people who use C# for scripting. Some things that were trivial before (office interop and invoking dynamic methods) are now even more trivial, while things that were impossible before remain impossible (e.g. writing a generic, high performance matrix class). 


    Here is a small test program to illustrate this:

        class Program
        {
            static Func<T, T, T> GetAdder<T>()
            {
                // declare the parameters
                ParameterExpression paramA = Expression.Parameter(typeof(T), "a"),
                    paramB = Expression.Parameter(typeof(T), "b");
                // add the parameters together
                BinaryExpression body = Expression.Add(paramA, paramB);
                // compile it
                Func<T, T, T> add = Expression.Lambda<Func<T, T, T>>(body, paramA, paramB).Compile();
                // call it
                return add;
            }
            static T GenericSum<T>(T[] items)
            {
                var adder = GetAdder<T>();
                var result = items[0];
                for (int i = 1; i < items.Length; i++)
                    result = adder(result, items[i]);
                return result;
            }
            static double DoubleSum(double[] items)
            {
                var result = items[0];
                for (int i = 1; i < items.Length; i++)
                    result = result + items[i];
                return result;
            }
            static void Main(string[] args)
            {
                var numbers = new double[1000000];
                for (int i = 0; i < numbers.Length; i++)
                    numbers[i] = i;
                var result = 0.0;
                {
                    var t0 = DateTime.UtcNow;
                    for (int i = 0; i < 1000; i++)
                        result += GenericSum(numbers);
                    var dt = DateTime.UtcNow - t0;
                    Console.WriteLine("dt=" + dt);
                }
                {
                    var t0 = DateTime.UtcNow;
                    for (int i = 0; i < 1000; i++)
                        result += DoubleSum(numbers);
                    var dt = DateTime.UtcNow - t0;
                    Console.WriteLine("dt=" + dt);
                }
                Console.ReadLine();
            }
        }

    These are the results on my 2.4GHZ quad core:

    x86
    dt=00:00:08.8770000
    dt=00:00:01.6440000

    x64
    dt=00:00:03.7410000
    dt=00:00:01.5200000
    Friday, November 14, 2008 9:47 AM
  • Charlie,

    Is anyone from the C# team able to respond to this topic? You made it sound like you thought you could get someone to respond but I've not heard any followup on this thread. Thanks.
    Wednesday, December 31, 2008 1:20 AM
  • There is a feature element that I would like in C# 4.0 since apparently it's not a closed issue yet. 

    I've been developing a set of complex mathematical classes that I found desirable to overload the various math operators.

    I found myself having to do this:

    public static ResultType operator + (ArgType1 left, ArgType2 right)
    {
      ResultType result = new ResultType();
      
      result.member = left.member + right.member;
     
      return result;
    }
      
    public static ResultType operator - (ArgType1 left, ArgType2 right)
    {
      ResultType result = new ResultType();
      
      result.member = left.member - right.member;
     
      return result;
    }

    public static ResultType operator + (ArgType2 left, ArgType1 right)
    {
      ResultType result = new ResultType();
      
      result.member = left.member + right.member;
     
      return result;
    }
      
    public static ResultType operator - (ArgType1 left, ArgType2 right)
    {
      ResultType result = new ResultType();
      
      result.member = left.member - right.member;
     
      return result;
    }
     

    Do you see the duplication? It would be nice to do something like this:

    public static ResultType operators +- (~ ArgType1 left, ArgType2 right)
     {
      ResultType result = new ResultType();

      result.member = left.member +- right.member;

      return result;
    }

    This simply uses a couple of tokens that generate the proper functions. The "+-", "*/", "+-*/", etc. are substituted with the proper operaters for each created function, and the "(~" token simply creates templates with the arguments exchanged. Obviously this is just a draft concept, but it is consistent with the autogenerated C# "+=", "-=", etc. operator functions that are created when you overload the corresponding mathematical or logical operators.
    Friday, May 15, 2009 7:39 PM
  • @Rüdiger Klaehn: It would probably help if you cached the delegate over all 1000 iterations... easy to do with a static field.

    @AvelWorldCreator: I wouldn't be hopeful... interesting idea, but I wouldn't expect much to change in 4.0 at this point except for bug fixes and optimisations

    Marc [C# MVP]
    Saturday, May 16, 2009 9:10 AM