none
How to merge lists in one big list? RRS feed

  • Question

  • I am trying  to merge many lists into one but without success. 

    Here's what i have tried, anyone can tell me what's wrong.

    List<double> tlist = new List<double>() {alist[a]};
    List<double> ttlist = new List<double>() {zlist[z]};
              
              var list = tlist.Concat(ttlist).ToList();
              List<double> ulist = tlist.Union(ttlist).ToList();
              
              Print(ulist);

    I tried with concat and union but i always get an error message:

    System.Collections.Generic.List`1[System.Double]

    I have to mention that those lists are from another lists like:

    List<double> alist = new List<double>() {xlist[y]};
            
              for(int a = 0; a < alist.Count; a++)
                {
             alist[a] = (alist[a] + counter) / (someOtherInt - counter);
    thank you

    Monday, July 8, 2019 11:44 AM

All replies

  • so your result should be a big List<double>() ? What about AddRange?

                var list1 = new List<double>();
                var list2 = new List<double>();
    
                list1.Add(1.2);
                list1.Add(2.3);
                list1.Add(3.4);
                list1.Add(4.5);
                
                list2.Add(5.6);
                list2.Add(6.7);
                list2.Add(7.8);
                list2.Add(9.0);
    
                list1.AddRange(list2);

    Monday, July 8, 2019 11:55 AM
  • I am not sure i understand because i am very new at c#.

    Why the  list1.Add(1.2);
                list1
    .Add(2.3);
                list1
    .Add(3.4);
                list1
    .Add(4.5);
               
                list2
    .Add(5.6);
                list2
    .Add(6.7);
                list2
    .Add(7.8);
                list2
    .Add(9.0);

    Did you mean:

    List<double> tlist = new List<double>() {alist[a]};
              List<double> ttlist = new List<double>() {zlist[z]};
              
              
                      var tlist = new List<double>();
                         var ttlist = new List<double>();

                        
                         tlist.AddRange(ttlist);
              
              Print(tlist);
              

    Sorry i dont get it.

    Monday, July 8, 2019 12:58 PM
  • Seems AddRange and Concat should both work.

    var list1 = Enumerable.Range(1, 5).Select(Convert.ToDouble);
    var list2 = Enumerable.Range(21, 5).Select(Convert.ToDouble);
    
    var bothTogether = list1.Concat(list2).ToList();

    .

    var list1 = Enumerable.Range(1, 5).Select(Convert.ToDouble).ToList();
    var list2 = Enumerable.Range(21, 5).Select(Convert.ToDouble).ToList();
    list1.AddRange(list2.ToArray());


    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    Monday, July 8, 2019 1:38 PM
    Moderator
  • `alist` is List<double>. I assume `zlist` is the same. In your very first call you are getting the `a` and `z` elements out of those lists. Those are doubles, not lists. Therefore your `tlist` and `ttlist` have a single element each. Probably not what you wanted. You also mentioned an error but you didn't print the actual error message. Seems to be related to List<double> so it sounds like one of your variables aren't typed the way you think.

    I'd ditch all the calls to ToList and even Concat/Union, you don't need them. As Christoph mentioned you can use AddRange to combine a list into an existing list. As Karen mentioned you can combine 2 lists into a new list via Concat or Union. Alternatively just use LINQ to combine them into IEnumerable<T> and then convert to list only when you need the final results.

    //Assumption 1: alist and zlist are IEnumerable<double> or List<double>
    //Assumption 2: You want a new list while leaving the original lists alone
    
    //Join them together
    var list = alist.Union(zlist);
    
    //Should have signature void Print ( IEnumerable<double> )
    Print(list);


    Michael Taylor http://www.michaeltaylorp3.net

    Monday, July 8, 2019 2:04 PM
    Moderator
  • I should explain more thoroughly in fact it began with another post i made. In this case my goal was to find the closest number from my variable fibo.

    Here's the post:

    https://social.msdn.microsoft.com/Forums/vstudio/en-US/bd65d71d-4128-4451-a6ab-8cb05eb189d0/how-to-extract-results-from-a-list-and-use-it-with-linq-?forum=csharpgeneral

    Unfortunately, i have been unable to solve that problem. I also need to merge all those lists and than try to find the closest number from my variable. So i decided to create that post to ask how i can solve the merging problem which seems similar to my closest number problem. Its like no LinQ solution can be applied.

    In this case i want to test the result of the list:

    List<double> alist = new List<double>() {xlist[y]};
            
              for(int a = 0; a < alist.Count; a++)
                {
             alist[a] = (alist[a] + counter) / (someOtherInt - counter);

    So i need to first merge all results of alist[a] , zlist[z], dlist[d] etc. There is near 50 of them.

    If i try that solution:

    var alist = Enumerable.Range(1, 5).Select(Convert.ToDouble);
              var zlist = Enumerable.Range(21, 5).Select(Convert.ToDouble);

              var bothTogether = alist.Concat(zlist).ToList();
                        
      it return this error:

    SampleDisplayBarsAgo.cs A local variable named 'alist' can not be declared in this scope, because it would change the meaning of 'alist', already used in a 'parent' or 'current' scope to designate something else CS0136 590 15

    If i try :

    var list = alist.Union(zlist);
              
              Print(list);

    i get those words in the output window and no error when compiling:

    System.Linq.Enumerable+<UnionIterator>d__67`1[System.Double]

    But you are right about Therefore your `tlist` and `ttlist` have a single element each.

    Actually the result from alist[a]  return a bunch of numbers in the output window 1.5, 1.6, 1.8 etc for exemple.

    All those numbers from each list need to be merge and than i find the closest number.

    If i get there one day.

    I hope my explanation are better this time.

    Thank you

    Monday, July 8, 2019 2:50 PM
  • [...]

              List<double> ulist = tlist.Union(ttlist).ToList();
              
              Print(ulist);

    I tried with concat and union but i always get an error message:

    System.Collections.Generic.List`1[System.Double]

    [...]

    This is not an error, but a special name of ulist list. Probably Print is not good. Show some details. In order to print the list, you can use a for loop.


    Monday, July 8, 2019 2:53 PM
  • Your code doesn't line up with your explanation. You keep saying `alist[a]` is a list but in your very first post you posted a simple foreach which sets each value. Let's save everyone a lot of time and just clarify the types. Please post the actual type declarations for your `alist`, etc. How they are initialized doesn't matter, just the types.

    If they are, indeed List<double> then the above code that was posted about union should combine them all into a single list. After that code runs look in the debugger at the new variable. Does it have the values you expect?

    If so then the issue is with your Print method. Post that code.


    Michael Taylor http://www.michaeltaylorp3.net

    Monday, July 8, 2019 2:56 PM
    Moderator
  • int barminus = Math.Abs(bar - ChartBars.ToIndex);

        for(int barIndex = ChartBars.GetBarIdxByX(chartControl, cursorPointX); barIndex <= ChartBars.ToIndex; barIndex++)
                    {
                     double closePrice = Bars.GetClose(barIndex);
            double highPrice = Bars.GetHigh(barIndex);
         double lowPrice = Bars.GetLow(barIndex); 
         double volumes = Bars.GetVolume(barIndex); 
            double average = ((newsum2+newsum)/newbar1);

         for (int i = 0; i <= barIndex; i++)
           {
          if(average < highPrice && average > lowPrice)
           {  
          int decrementBy = barminus;
                                List<int> xlist = new List<int>() {barIndex};

             for(int y = 0; y < xlist.Count; y++)
                {
             xlist[y] = (xlist[y] - decrementBy);
              {
          //total0          
                               List<double> zlist = new List<double>() {xlist[y]};

             for(int z = 0; z < zlist.Count; z++)
                {
             zlist[z] = (zlist[z] /= someOtherInt);

          //total
           List<double> alist = new List<double>() {xlist[y]};

              for(int a = 0; a < alist.Count; a++)
                {
             alist[a] = (alist[a] + counter) / (someOtherInt - counter);

    Well sorry i am a newbie sometimes i dont use the right words. I hope this is what you asked.

                                           

    Print(alist) return System.Collections.Generic.List`1[System.Double]

    Print(alist[a]) return numbers result from the calculation ex (1.2,1.3,1.56)

    In the end i want to merge alist[a], blist[b], etc.

    Sorry for the confusion. 


    • Edited by Frankdot Monday, July 8, 2019 3:34 PM
    Monday, July 8, 2019 3:20 PM
  • `xlist` and `zlist` are List<double>, they are not lists of lists.

    double element1 = xlist[0];

    You keep saying you want to merge `alist[a]` and `blist[b]` but they aren't lists. They are 2 distinct double values like 3.4 and 5.7. Do you want to put them into a single list? If so then:

    //List has 2 double values - alist[a] and blist[b]
    var newList = new List<double>() {
       alist[a],
       blist[b]
    };

    `newlist` will have 2 double values now. Imagine it this way.

    //A list of doubles
    List<double> alist = new List<double>();
    
    //List now contains a single element with value 10
    alist.Add(10);
    
    //List now contains 2 elements - 10 and 20
    alist.Add(20);
    
    //Get the first element from the list
    double is10 = alist[0];
    
    //Get the second element from the list
    double is20 = alist[1];
    
    

    The earlier code given that combines the lists (not lists[?]) will work for lists but not elements within the list. To add to an existing list use `Add`. To create a new list use the `new` operator to create the new list like shown earlier and add the element(s) you want to it.

    Please post the code for `Print`.


    Michael Taylor http://www.michaeltaylorp3.net

    Monday, July 8, 2019 4:38 PM
    Moderator
  • If  i add that part of the script:

    var newList = new List<double>() { alist[a], blist[b]};

    List<double> alist = new List<double>();

    alist.Add(10);

    alist.Add(20);double is10 = alist[0];

    double is20 = alist[1];

    Print(is10);

    Its crashing the program (ninjatrader; charting program)

    I dont understand the add value part of 10 and 20.

    Monday, July 8, 2019 5:24 PM
  • That was just demoing how you'd add individual double values to an existing list. Since your original `alist[a]` is a double it was just showing what is happening. You don't need to add that to your code.

    The fact that your code compiles leads me to believe that `Print` is accepting an object. Please post the code for this method.


    Michael Taylor http://www.michaeltaylorp3.net

    Monday, July 8, 2019 5:28 PM
    Moderator
  • I'm sorry i dont get it at all. 

    This will return System.Collections.Generic.List`1[System.Double]

    var newList = new List<double>() { alist[a], blist[b]};

    As for Print i dont know what you want. Print is a method from  ninjatrader program.

    You just write Print(); and it will send the info in the output window.

    If there is something else some method behind it i will have to ask NT people.

    Cause if the idea is to do that:

     var list = alist[a]Concat(blist[b]).ToList(); it wont work.

    Even if i create some double value it wont work.

    ex:  

    double myalist = alist[a]

    and input var list =myalistConcat(myblist).ToList(); it wont work.

    ty

    Monday, July 8, 2019 6:16 PM
  • Maybe try this too:

       var list = myalist.Concat( myblist ).ToList( );

       string s = string.Join( ", ", list );

       Print( s );

    Monday, July 8, 2019 6:25 PM
  • Then Print likely has a signature of `void Print ( object value )`. Therefore it calls `ToString` on the object. In the case of most types that would be the type name and hence why you are getting your `List<double>` message. Therefore `Print` isn't going to be useful to you because you cannot change what is being printed because you cannot change `ToString` on List<T>.

    What is the actual problem you're trying to solve here? Are you just trying to concatenate some lists together? If so then how does the code posted earlier not work for you?

    var list = alist.Union(blist);

    `list` would be a combination of everything in `alist` and everything in `blist`. Note, Print WILL NOT display the correct results. There is nothing you can do about that. If you look at `list` in the debugger then it will have the concatenated values. If you were to call `foreach` on `list` then you'd see it had each of the values as well. If this is not working for you then please post just this fragment of your code along with exactly what you're seeing in the debugger window.

    If instead you need `Print` to display the values for you then you cannot use List<T> directly. Instead you'll need to create a wrapper type that contains the List<T> and then override the ToString method to print out what you want. Then when you pass the object to `Print` it should work.

    public class MyData
    {
       //Completely making up stuff here
       public List<double> AList { get; set; } = new List<double>();
    
       public List<double> BList { get; set; } = new List<double>();
    
       public override string ToString ()
       {
           //Concatenate the lists
           var items = AList.Union(BList);
    
           //Now print them out separated by commas
           return String.Join(", ", items);
       }
    }

    Now use this type in your code.

    var data = new MyData();
    data.AList = alist;
    data.BList = blist;
    
    Print(data);


    Michael Taylor http://www.michaeltaylorp3.net

    Monday, July 8, 2019 6:29 PM
    Moderator
  • One thing before we get to CoolDadTx i would like to mention Viorel solution did work for a few minutes before it crashes. It last about 2 minutes if i move the indicator just a little NT program crash.

    Here's the result in the output window:

    6,70454545454545, 0,515151515151515
    6,70454545454545
    0,515151515151515
    6,71212121212121, 0,522727272727273
    6,71212121212121

    6,75, 0,560606060606061
    6,75
    0,560606060606061

    Well i cannot post the entire result as a file but it gives you an idea. We're getting close.

    As you can see Viorel solution bring result next to each other.

    While the result  below is just Print(alist[a]) Print(blist[b]);

    I will now try Cool and i'm back in the next post.

    Monday, July 8, 2019 7:03 PM
  • I am not great in c# but from what the limit Ninjatrader impose me i dont think i can implement that solution.

    I get multiple errors try compiling. In fact so many i dont know which one start with.

    If it could be done without getting into the public maybe...

    That part bugs 

    public List<double> AList { get; set; } = new List<double>();

      
    public List<double> BList { get; set; } = new List<double>();

    If its a Print problem that would mean that it does the concat.

    But if i cant see the numbers in the output window i am blind and cant test. 

    If its a Print issue why cant i apply the solution to get the closest number.

    List<double> list = new List<double>() { alist[a] };
    var minDistance = list.Min(no => Math.Abs(fibo- no));
    var closest = list.First(no => Math.Abs(fibo- no) == minDistance);
    {
    Print(closest);
    }

    After all it should get me one number the closest and instead it return the entire list.


    • Edited by Frankdot Monday, July 8, 2019 7:35 PM
    Monday, July 8, 2019 7:17 PM
  • Are you limited to the contents of a single method then? That is what it sounds like. In that case stick with your list combined with `Union` and then instead of passing the list to `Print` pass the `String.Join` that I put into the `ToString` method or that `Viorel` showed. Either should work (with the correct variables).

    var list = alist.Union(blist);
    
    var output = String.Join(", ", list);
    Print(output);


    Michael Taylor http://www.michaeltaylorp3.net

    Monday, July 8, 2019 8:03 PM
    Moderator
  • Thank you for now its working without crashing.

    But like i said i want to merge more than 2 lists. Examples on the net always show the union of two lists.

    How do i get to merge alist, blist, clist, dlist, glist, llist etc.

    Can we do 

    var list1 = list.Union(clist);   ?

    there must be a more productive way to do that?

    I just got another problem if i try to narrow the number of returns it send an error message:

    var list = alist.Union(blist);
              
              var list1 = list.Union(clist);

              var output = String.Join(", ", list1);
              
               if (Math.Abs(fibo - output) < 0.01)
              
               Print(output.ToString("N2"));  

    NinjaScript File Error Code Line Column
    SampleDisplayBarsAgo.cs Operator '-' can not be applied to operands of type 'double' and 'string' CS0019 595 25

    NinjaScript File Error Code Line Column
    SampleDisplayBarsAgo.cs Overloaded method best matching 'string.ToString (System.IFormatProvider)' has invalid arguments CS1502 597 18

    NinjaScript File Error Code Line Column
    SampleDisplayBarsAgo.cs Argument 1: Can not convert 'string' to 'System.IFormatProvider' CS1503 597 34

    I'm not sure ill be able to apply this solution to the new list:


    var minDistance = output.Min(no => Math.Abs(fibo- no));
    var closest =output.First(no => Math.Abs(fibo- no) == minDistance);

    actually it returns a number that has nothing to do with the lists.

    like 44 when it should be 0.12


    to be precise it returns 44 if i replace var by double otherwise it returns a bunch of dots. any idea why?
    • Edited by Frankdot Tuesday, July 9, 2019 12:48 PM
    Tuesday, July 9, 2019 11:47 AM
  • I don't know how you define productive but to me I measure it in terms of runtime speed and memory usage. In terms of that calling Union on multiple lists is very productive. LINQ uses deferred execution so it doesn't actually bother to enumerate anything until you need it. If you, for example, only query the first 10 items from your list then it doesn't waste the time of actually "unioning" anything after that. Additionally since it returns IEnumerable<T> there is no extra memory allocation for the new list. It simply pulls things from the initial list. It is basically a wrapper around the underlying lists so it is very performant.

    //Doesn't actually join them until you need the values
    //Doesn't need to create any temporary memory to store the values, it enumerates the underlying lists directly
    var list1 = alist.Union(blist).Union(clist);

    Having said that the caveat is that if the underlying lists are changing then you'll have issues if you try to enumerate them. To force an execution and allocation of memory you can use ToList on the end. This should be done only if needed though as it defeats the benefits of LINQ.

    //Force enumeration of all the items and conversion to a list
    //Now have 2 copies of the items in memory but the underlying
    //lists can be changed without impacting the new list
    var list1 = alist.Union(blist).ToList();

    CS0019 

    'output' is a string because it is the concatenation of your list. You needed to do that for the call to Print but you shouldn't be using it elsewhere. Personally, unless you need this list elsewhere, just put it directly in the Print call so you won't use it.

    Print(String.Join(", ", list1));


    As for the value on the right side of that subtraction I have no idea what you should use. You have a list of doubles. You're algorithm will need to determine which of those values (if any) to use for that. It is a business question at this point.

    The next 2 errors are related to the Print call. You are passing a string to the method and specifying N2. But `output` isn't a double value. It is a list of the doubles you had. If you want to print out a specific value (probably the one from your calculation) then you can pass it with a call to `ToString`.

    var list1 = list.Union(clist);
    
    //I have no idea which of the items in your list1 you want to be "output" - that is a business decision, we'll use the first element here
    var output = list1[0];  //Double
    
    if (Math.Abs(fibo - output) < 0.01)
    
    Print(output.ToString("N2"));


    The remainder of your question, and actually most of this post, is really a lack of understanding of typing in C# (or any structured language). You're going to need to read up on this stuff as the forums cannot teach this to you.

    In a nutshell every variable has 1 and only 1 type (e.g int, string, double) that cannot be changed after declaration. If you are not very clear on what type each of your variables are then you're going to have the kind of issues you're running into. Using `var` is a shorthand for folks who understand typing. In your case I don't think you fully understand it so I'm going to recommend you stop using `var` and start using the actual type. This won't impact the generated code in any way but it will generate compiler errors that are more clear for you.

    Case in point `minDistance` could be an int or double. It depends upon the types of `fibo` and `no`. They are being subtracted so type coercion rules kick in. Type coercion is defined here. Let's assume `fibo` is a double then double - double is a double. You then call Math.Abs(Double) which returns a double. Since that is the return value of the lambda being passed to Min then Min returns a double. So `minDistance` is a double. If you hover over this variable in the IDE it should show you that type. To make sure you get it and to detect issues if you're wrong (ignoring type coercion) then change the type to be explicit for now.

    double minDistance = output.Min(no => Math.Abs(fibo - no));

    But there is one very big issue here, `output` isn't your list of numbers. It is that string that you stuffed all the values into, comma separated. Once again, getting rid of the output variable would break this code. You should be using your combined list variable `list1` instead. Now it will enumerate the doubles in your unioned list looking for the minimal value. But note that Min returns the smallest value returned, not the smallest value in your list. If you want the smallest value in your list then call Min with no parameters. In your case `minDistance` would be the smallest value returned by `Math.Abs` which may or may not be what you expected.

    Moving on to the `First` call. It returns the first value that has that absolute value. You're basically enumerating the list again which seems inefficient to me. But it will return (once you use `list1`) a double as well. Once you get this code working then there are ways to optimize it. For example you could find the min distance and actual value by just enumerating through the list once.

    Final sample

    //Generate a couple random lists
    List<double> alist = GenerateList(10);
    List<double> blist = GenerateList(10);
    List<double> clist = GenerateList(10);
    
    //Combine them
    IEnumerable<double> list = alist.Union(blist).Union(clist);
    
    double fibo = 4.5;
    
    //Find the first value that has the min abs from fibo
    double minDistance = list.Min(n => Math.Abs(fibo - n));
    double closest = list.First(n => Math.Abs(fibo - n) == minDistance);
    
    //An optimized version that requires one enumeration
    var closestValue = (from no in list
                        let distance = Math.Abs(fibo - no)
                        orderby distance
                        select new { Value = no, Distance = distance }
                        ).FirstOrDefault();
    
    //minDistance == closestValue.Distance;
    //closest = closestValue.Value;


    Michael Taylor http://www.michaeltaylorp3.net

    Tuesday, July 9, 2019 2:04 PM
    Moderator
  • Thank you for all your help and all who posted.

    The quest to find the closest number is still on. I tried your solution:

    var closestValue = (from no in list
                       
    let distance = Math.Abs(fibo - no)
                       
    orderby distance
                       
    select new { Value = no, Distance = distance }
                       
    ).FirstOrDefault();

    But its returning all the list. It gives a value for each numbers returned. Like it was looking for the closest number for each number and  not as a whole list. Strange...after a moment it crash.

    Oh well i am going to keep researching this. As you noticed i still have a lot to learn.

    But at least the concat of all my list is a success. 


    Tuesday, July 9, 2019 2:54 PM
  • "But its returning all the list. "

    That is not possible if `list` is List<double>.  FirstOrDefault would return a double. If you're getting back a list then `list` is not List<double> it is List<List<double>> which was the conversation that happened earlier in the discussion. Mousing over the variable in the IDE will show you the exact type.


    Michael Taylor http://www.michaeltaylorp3.net

    Tuesday, July 9, 2019 3:16 PM
    Moderator
  • Keep in mind this Ninjatrader charting program. ninjatrader.com

    I'm not sure what you mean by IDE. But everything is in double and return doubles. I used var because i took the closest script at stack overflow. Like i said change all the var for double return dots in the output window and no doubles. 

    I took the list from: 

    var list = alist.Union(blist);
              var list1 = list.Union(clist);
                      var output = String.Join(", ", list1);

    maybe i should of done it with:

    IEnumerable<double> list = alist.Union(blist).Union(clist);


    • Edited by Frankdot Tuesday, July 9, 2019 4:30 PM
    Tuesday, July 9, 2019 4:29 PM
  • I have no knowledge of Ninjatrader and we don't provide support for that product here. When I say IDE I'm talking about Visual Studio or Visual Studio Code which is where we write C# code. If you're using another editor then we aren't going to be able to provide you help on how to use its IDE.

    Nevertheless your first block of code and your second block of code is the same thing. Change to the second version and see if you get a compiler error about not being able to convert from the union stuff to IEnumerable<double>. If it works then I don't know what the dots mean in your tool. If it fails then `alist` and friends aren't List<double>.


    Michael Taylor http://www.michaeltaylorp3.net

    Tuesday, July 9, 2019 4:51 PM
    Moderator
  • Here's the result fibo = 1.48 and you can see the returns.

    logically there should be only one return 1.41

    it crash ninjatrader not long after and its the same result with IEnumerable

    1,48

    { Value = 1,41071428571429, Distance = 0,0698568753343787 }
    { Value = 1,61607142857143, Distance = 0,135500267522764 }

    { Value = 1,625, Distance = 0,144428838951336 }


    I got another issue when union like this:

    var list = zlist.Union(alist);
              var list0 = list.Union(blist);
              var list1 = list0.Union(clist);
              var list2 = list1.Union(dlist);
              var list3 = list2.Union(elist);
              var list4 = list3.Union(flist);
              var list5 = list4.Union(glist);
              var list6 = list5.Union(hlist);
              var list7 = list6.Union(jlist);
              var list8 = list7.Union(klist);
              var list9 = list8.Union(mlist);
              var list10= list9.Union(nlist);
              var list11 = list10.Union(olist);
              var list12 = list11.Union(plist);
              var list13 = list12.Union(qlist);
              var list14 = list13.Union(rlist);
              var list15 = list14.Union(slist);
              var list16 = list15.Union(zzlist);

    It produce a result where numbers are next to each other instead of under.

    0,650406504065041, 3,95121951219512, 1,95121951219512, 1,31707317073171, -0,00975609756097561, 0,790243902439024, -0,0487804878048781, -0,016260162601626, 0,390243902439024, 0,336134453781513, 1,03846153846154, 0,512820512820513, 0,680672268907563, -0,00625, 0,50625, -0,0128205128205128, -0,00840336134453781, 0,25, 0,0125

    How to avoid this?

    • Edited by Frankdot Tuesday, July 9, 2019 5:16 PM
    Tuesday, July 9, 2019 5:03 PM
  • The crash you're going to have probably go to the Ninjatrader forums for. We have no knowledge of that app here. At a minimum you'll need the error message. The only thing I can think of is that the underlying list(s) you're reading are changing and therefore when you try to enumerate them it'll crash. To resolve that you have to copy the lists. You can do that by calling ToList on the resulting items.

    var list = zlist.Union(alist)
                    .Union(blist)
                    .Union(clist)
                    ...
                    .ToList();

    However another concern is that you're loading a lot of data into memory and therefore may be running out of memory. You might need to do your processing in batches. This wouldn't be a question for this post though.

    It produce a result where numbers are next to each other instead of under.

    Again, that is because it is just calling ToString on the result. Since your String.Join used a comma then that is what you get. If you want each value on a separate line then try using a newline. Unfortunately newline is different for different environments. You can try using Environment.NewLine and see if that works but it is possible the Print method doesn't support it.

    Print(String.Join(Environment.NewLine, list));


    Michael Taylor http://www.michaeltaylorp3.net

    Tuesday, July 9, 2019 9:46 PM
    Moderator
  • In fact there is no error message it just crash.

    Yes i am processing a lot of data. This is an indicator for futures market. It analyze data tick by tick over many minutes on many days. That would explain the lag but not the crash. If i just Print() the return of the formulas i get slow, lagging, jumpy indicator in the screen but no crash. 

    I was testing everything in var and it returns one number but a round one like 32 when in fact it should be 0.035 or whatever. 

    The Xlist  from  List<double> zlist = new List<double>() {xlist[y]}; return a Int. 

    The result from the transformation:

    for(int z = 0; z < zlist.Count; z++)
                {
             zlist[z] = (zlist[z] /= someOtherInt);  returns a double

    I was trying this morning to int the list instead of var and keep the rest in double but no luck it gives an error compiling. 

    NinjaScript File Error Code Line Column
    SampleDisplayBarsAgo.cs Can not implicitly convert type 'System.Collections.Generic.IEnumerable <double>' to 'int' CS0029 589 22

    injaScript File Error Code Line Column
    SampleDisplayBarsAgo.cs' int 'does not contain a definition for' Union 'and the best overload of the extension method' System.Linq.Queryable.Union <TSource> (System.Linq.IQueryable <TSource>, System.Collections. Generic.IEnumerable <TSource>) 'contains invalid arguments CS1928 590 23

    NinjaScript File Error Code Line Column
    SampleDisplayBarsAgo.cs Instance Argument: Can not convert 'int' to 'System.Linq.IQueryable <double>' CS1929 590 23

    union the lists slow the indicator and ninjatrader but it does not affect the memory or the rest of the laptop or windows 10.

    Union for now is unmanageable i had to close the program in task manager because it was freezing.

    I couldn't try Print(String.Join(Environment.NewLine, list));

    Its not important the way i see the data if its side by side or under. As long as i get the closest number from fibo.

    It was better (less slow) when i was Printing() one by one the list and use if (Math.Abs(fibo - blist[b]) < 0.01) to narrow down the list but it does not return the closest number.





    • Edited by Frankdot Wednesday, July 10, 2019 12:38 PM
    Wednesday, July 10, 2019 12:17 PM
  • I think you're dealing with too much data if Union is slow. If you're talking about thousands of items in each list then this is beyond what we would try to do in code. It would be better to handle this in the database and then render the results. If that isn't possible then you're going to need to partition the data so you aren't trying to work with all of it at once. Perhaps parallel if Ninjatrader supports it. I think you need to talk to the program devs to see how to handle large amounts of data.

    Here's the logical code for Union. As you can see it doesn't do anything expensive but it will get progressively worse the larger the arrays. There isn't any way to speed this up. If you try to enumerate millions of rows it's going to take a minute.

    foreach (var item in first)
       yield return item;
    foreach (var item in second)
       yield return item;

    The only thing Union does that is a little different is that it returns distinct values. So if you have dups it filters them out. If you want dups then use Concat instead. Performance wise it'll likely be similar though.

    As for the int I don't know what to tell you. Your lists are List<double> so elements in that are doubles. You aren't going to get ints unless you cast down. The only thing I can think of is whatever you're using to view the data is rounding the value to an integer. You can use ToString("N2") to round the double values when you display them but since I know nothing about the Ninjatrader program it may be something with their environment. Again, you should contact them.

    I think we've provided as much help as we can on your original problem. I think some of the issues you're seeing could be related to the data you're working with and the tool. I recommend you contact them to see if they can help you with the remaining work. The code itself, in a pure C# app, would be fine so this isn't a C# issue at this point I believe.


    Michael Taylor http://www.michaeltaylorp3.net

    Wednesday, July 10, 2019 2:00 PM
    Moderator
  • Your help is appreciated and i agree i have to deal with the limits of Nt.

    I already ask them about data being slow to process and they said there's not much to do considering the large amount of data the indicator is dealing with.

    I was hoping by solving the issue about the closest number it would of resolve the slow issue. Because if there's one number returned it runs smoothly.

    Actually if i dont open the Output window to see the result print() it works perfectly fine.

    But at some point i have to check on the returns to be sure it gives the right results. 

    Ill ask them again now that we have a new approach maybe one of their tech will think of something.

    Thanks again


    • Edited by Frankdot Wednesday, July 10, 2019 2:22 PM
    Wednesday, July 10, 2019 2:19 PM