locked
Limit of Program's size into RAM RRS feed

  • Question

  • Hello,

    I have a program who show a voxel cube 498*498*498 (1,9 GB)  and *4 for coordinate (x, y, z and color), it's work for this configuration and here's my problem

    When i want more voxel (500*500*500 2GB), windows say to me: is not an application Win32, i'm on windows seven ultimate x64, 5GB ram.

    What's the solution ?





    • Edited by shaynox Friday, October 3, 2014 4:02 AM
    Friday, October 3, 2014 3:57 AM

Answers

All replies

  • for a 32 bit application , the max amount ram that you can access is 4 GB and in that 2 GB for usermode and another 2GB for the Kernel, application comes under usermode hence 2GB.

    so i think your application is compiled as 32 bit, that's why you are not able to access more than 2GB of RAM, try to compile it as a 64bit application and try.

    Friday, October 3, 2014 4:58 AM
  •  No, i assemble my program in 64 bit mode


    • Edited by shaynox Friday, October 3, 2014 5:08 AM
    Friday, October 3, 2014 4:59 AM
  • Here's two executable of my program, one with 498*498*498 cube and other 500*500*500 cube.

    http://wqaxs36.perso.sfr.fr/engine.zip


    This programm need AVX support, intel CPU
    • Edited by shaynox Friday, October 3, 2014 5:10 AM
    Friday, October 3, 2014 5:05 AM
  • When i want more voxel (500*500*500 2GB), windows say to me: is not an application Win32

    Make sure that you have the linker option:

    System->Enable Large Addresses

    set to "Support Addresses Larger Than 2 Gigabytes (/LARGEADDRESSAWARE)"

    - Wayne

    Friday, October 3, 2014 1:40 PM
  • don' work :/
    Friday, October 3, 2014 2:18 PM
  • Is your voxel array static or dyanmically allocated?  Try dynamically allocating it to see if that helps.
    Friday, October 3, 2014 2:24 PM
  • i declare array like this:

    (ASM):

    [section .bss use64]
        Cube2_game resd 498*498*498*4

    (C):

        static float    voxel_cube[498* 498 *498 *4];




    • Edited by shaynox Friday, October 3, 2014 3:12 PM
    Friday, October 3, 2014 2:57 PM
  • Here's the disassembly code of C project:

    voxel_cube    DB ?    ; pad
        ORG $+1976095870    ; pad
        DB ?    ; pad

    So yes, we can say he build voxel cube dynamicaly way


    • Edited by shaynox Friday, October 3, 2014 3:06 PM
    Friday, October 3, 2014 3:05 PM
  • With malloc function:

    float *voxel_cube;

    voxel_cube = (float*)malloc(498 * 468 * 498 * 4);

    ...

    It's do this:

    Unhandled exception at 0x000000013FC5112A in 3D engine(SDL 2).exe: 0xC0000005: Access violation writing location 0x000000001D712000.

    :D


    • Edited by shaynox Friday, October 3, 2014 3:22 PM
    Friday, October 3, 2014 3:21 PM
  • Try another malloc:

        malloc( 498 * 498 * 498 * 4 * sizeof(float) )


    and check the return value, which should not be null.
    • Edited by Viorel_MVP Friday, October 3, 2014 6:10 PM
    • Marked as answer by shaynox Friday, October 3, 2014 8:48 PM
    Friday, October 3, 2014 6:05 PM
  • It's works ! thanks (more 500*500*500*4).

    But do you know why it don't work with static data ?


    • Edited by shaynox Friday, October 3, 2014 6:18 PM
    Friday, October 3, 2014 6:15 PM
  • don' work :/


    What does that mean? Are you still getting the *same* error as you said originally?

    >"windows say to me: is not an application Win32"

    Or do you now get a *different* error? Apparently it's different.

    The exes in your zip file were *not* built with Large Address Aware Set, but the dll was.
    You must use that option to access > 2GB even on Win64.

    Your later posts suggest that you are now having *another* issue which is causing errors
    at run time, such as:

    >Unhandled exception at 0x000000013FC5112A in 3D engine(SDL 2).exe: 0xC0000005: Access violation writing location

    When switching focus from one error to another, always make it clear that the problem has changed.

    There is a limit of 2GB of *static* data even in Win64 applications.

    There is a limit of 2GB of *dynamic* data in Win64 applications if you don't build with
    Large Address Aware set,

    - Wayne



    Friday, October 3, 2014 6:29 PM
  • But do you know why it don't work with static data ?


    Study these links:

    Memory Limits for Applications on Windows
    https://software.intel.com/en-us/articles/memory-limits-applications-windows

    What is the maximum size of an array (globally) in C++ / C ?
    https://answers.yahoo.com/question/index;_ylt=A0LEVxgz7C5UcyAAtntXNyoA;_ylu=X3oDMTE0b3I1MmZ1BHNlYwNzcgRwb3MDMTIEY29sbwNiZjEEdnRpZANWSVAzODBfMQ--?qid=20120419190312AA8bG9j

    Memory Limits for Windows and Windows Server Releases
    http://msdn.microsoft.com/en-us/library/aa366778.aspx

    RAM, Virtual Memory, PageFile and all that stuff
    http://members.shaw.ca/bsanders/WindowsGeneralWeb/RAMVirtualMemoryPageFileEtc.htm

    - Wayne

    • Marked as answer by shaynox Friday, October 3, 2014 9:15 PM
    Friday, October 3, 2014 6:41 PM
  • Hello,

    I have a program who show a voxel cube 498*498*498 (1,9 GB)  and *4 for coordinate (x, y, z and color), it's work for this configuration and here's my problem

    When i want more voxel (500*500*500 2GB), windows say to me: is not an application Win32, i'm on windows seven ultimate x64, 5GB ram.

    What's the solution ?





    You will find that for that type of project, you have no real practical option except for installing a lot of RAM. I have seen this sort of voxel work in the past and its a real memory pig at the best of times.

    Some game engines use that for shaders but with smaller dimentions



    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Friday, October 3, 2014 7:27 PM
  • Some game engines use that for shaders but with smaller dimentions

    <off-topic start>

    Vegan -

    I'm surprised you haven't jumped into this thread:

    Can someone help write the code/program for this formula for Visual Studio?
    http://social.msdn.microsoft.com/Forums/vstudio/en-US/1915c238-cb02-426e-8a34-1ed606416e2b/can-someone-help-write-the-codeprogram-for-this-formula-for-visual-studio?forum=vbgeneral

    It looks like the type of diversion you relish.

    <off-topic end>

    - Wayne

    Friday, October 3, 2014 7:42 PM
  • Some game engines use that for shaders but with smaller dimentions

    <off-topic start>

    Vegan -

    I'm surprised you haven't jumped into this thread:

    Can someone help write the code/program for this formula for Visual Studio?
    http://social.msdn.microsoft.com/Forums/vstudio/en-US/1915c238-cb02-426e-8a34-1ed606416e2b/can-someone-help-write-the-codeprogram-for-this-formula-for-visual-studio?forum=vbgeneral

    It looks like the type of diversion you relish.

    <off-topic end>

    - Wayne

     I have a voxel shader, which is what I expected the OP was looking to emulate



    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Friday, October 3, 2014 7:44 PM
  • Sorry if i don't put a good explanation, i'm french :/


    Thanks for your urls
    • Edited by shaynox Friday, October 3, 2014 7:50 PM
    Friday, October 3, 2014 7:49 PM
  • Sorry if i don't put a good explanation, i'm french :/

    Merci pour intensifier, dans les moteurs de jeu voxel terme est utilisé pour le pixel shader. 

    Pouvez-vous me dire ce que vous avez l'intention de le faire, je parle assez bien français.


    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Friday, October 3, 2014 7:53 PM
  • Je désire réaliser un moteur 3D avec 100% utilisation du CPU et 0% du GPU.


    Et utilisation de voxel/atom/coord/pixel.

    Au moins mettre a profil les technologies (vectorisation, pouvoir déplacer de grosse donnée (512 bit) du CPU au lieu ceux du GPU







    • Edited by shaynox Friday, October 3, 2014 8:01 PM
    Friday, October 3, 2014 7:55 PM
  • Some game engines use that for shaders but with smaller dimentions

    <off-topic start>

    Vegan -

    I'm surprised you haven't jumped into this thread:

    Can someone help write the code/program for this formula for Visual Studio?
    http://social.msdn.microsoft.com/Forums/vstudio/en-US/1915c238-cb02-426e-8a34-1ed606416e2b/can-someone-help-write-the-codeprogram-for-this-formula-for-visual-studio?forum=vbgeneral

    It looks like the type of diversion you relish.

    <off-topic end>

    - Wayne

    -1 to you wayne, for I was able to spot the issue with the original post immediately



    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Friday, October 3, 2014 8:03 PM
  • Étant donné que vous souhaitez utiliser le CPU est bien, mais la vraie question est principales exigences de mémoire. Rappelez-vous le GPU est seulement un co-processeur comme le vieux x87 (plus ou moins). 

    Compte tenu de la taille de votre matrice, vous n'avez pas d'options réelles en dehors de l'aide d'un programme pixel shader de haut niveau. 

    Les jeux qui utilisent cette technique utilisent maintenant les petites 100x100x100 grilles autour d'un objet qui est tessalated et ombragés avec AO etc


    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Friday, October 3, 2014 8:06 PM
  • -1 to you wayne, for I was able to spot the issue with the original post immediately

    As usual. I have no idea what you're talking about. (And probably vice-versa.)

    My off-topic post that you quoted was suggested to you as it appears to have content
    that you enjoy: game programming, mathematics, etc.

    If you're actually referring to this thread, then you've missed the point of it completely.

    - Wayne

    Friday, October 3, 2014 8:09 PM
  • je parle assez bien français.

    Et moi aussi - but these are English language forums and threads/posts should be in English
    here. There are French forums. e.g. -

    http://social.msdn.microsoft.com/Forums/vstudio/fr-fr/home?forum=visualcplusfr&filter=alltypes&sort=lastpostdesc

    - Wayne

    Friday, October 3, 2014 8:11 PM
  • -1 to you wayne, for I was able to spot the issue with the original post immediately

    As usual. I have no idea what you're talking about. (And probably vice-versa.)

    My off-topic post that you quoted was suggested to you as it appears to have content
    that you enjoy: game programming, mathematics, etc.

    If you're actually referring to this thread, then you've missed the point of it completely.

    - Wayne

    The inquirer is using 3D so that how I spotted the voxel.

    I am sorry to use French, but the OP is francophone so I felt it would be easier for him if I used that.

    Now it's clearer what is going on

    This is C++ here, so DX is a part of the Windows SDK so I can step up

    your reference was to some Visual Basic topic, which is away from C++ which is the only language I use



    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Friday, October 3, 2014 8:14 PM
  • je parle assez bien français.

    Et moi aussi - but these are English language forums and threads/posts should be in English
    here. There are French forums. e.g. -

    http://social.msdn.microsoft.com/Forums/vstudio/fr-fr/home?forum=visualcplusfr&filter=alltypes&sort=lastpostdesc

    - Wayne

    That would likely not get much help, not many C++ gurus there

    sadly the world for C++ is squarely lang=en_us



    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Friday, October 3, 2014 8:16 PM
  • Sans doute, mais comme je suis qu'au début du stade de développement, je ne migrerais pas vers la carte graphique.

    Mais est-ce que le GPU économise des calculs en éliminant les pixel non visible, ou il est obligé de calculé tout les vertex de la scène rendu à l’écran ?

    Friday, October 3, 2014 8:21 PM
  • I am sorry to use French, but the OP is francophone so I felt it would be easier for him if I used that.

    Yes, I'm sure it helps the OP. But please consider *also* posting the same content in English
    so the majority of readers of the English-language forums can follow along. (Without having
    to resort to web translators.)

    - Wayne

    Friday, October 3, 2014 8:23 PM
  • Sans doute, mais comme je suis qu'au début du stade de développement, je ne migrerais pas vers la carte graphique.

    Mais est-ce que le GPU économise des calculs en éliminant les pixel non visible, ou il est obligé de calculé tout les vertex de la scène rendu à l’écran ?

    Vous déménagez définitivement dans une zone très avancé de développement avec HLSL et DX11 
    Si vous voulez éviter le GPU, je dois mentionner que la construction 64 bits utilise SSE2 qui requinquer calculs


    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Friday, October 3, 2014 8:25 PM
  • I am sorry to use French, but the OP is francophone so I felt it would be easier for him if I used that.

    Yes, I'm sure it helps the OP. But please consider *also* posting the same content in English
    so the majority of readers of the English-language forums can follow along. (Without having
    to resort to web translators.)

    - Wayne

    I agree, but I am here to help the OP

    Sure he could learn English, but that takes a lot of hard work



    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Friday, October 3, 2014 8:26 PM
  • Et bien, pour l'instant mon code est construis autour de la technologie SSE, donc je garde espoir en ce qui concerne l’accélération de vitesse a travers les technologies AVX et AVX2.

    • Edited by shaynox Friday, October 3, 2014 8:42 PM
    Friday, October 3, 2014 8:27 PM
  • Mais est-ce que le GPU économise des calculs en éliminant les pixel non visible, ou il est obligé de calculé tout les vertex de la scène rendu à l’écran ?

    This thread is now moving into an entirely different subject area than the original post
    and the Subject - which was how to use large arrays without run time errors. That original
    issue has been addressed and resolved. This thread should be marked as "Answered" and closed.

    Discussion of other aspects of the program's development should be posted to new threads,
    preferably one issue/problem per thread. A single thread should not be used to address
    *all* of the problems related to a program's development.

    - Wayne

    Friday, October 3, 2014 8:32 PM
  • when I read the first few answers, I realized that all the answers were useless to misleading

    that was why I stepped up, clearly he was using Win32 but as a 64-bit build. everybody missed that too.

    I can help the OP do a voxel shader if he really needs one, or it can be done with conventional linear algebra too

    I agree that new threads for new issues should be done, but that is not how most here post

    so I focus on the first post, the rest can tag along



    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Friday, October 3, 2014 8:38 PM
  • when I read the first few answers, I realized that all the answers were useless to misleading


    Your ignorance or ineptitude is showing again. Before you had posted even one reply to
    this thread the errors which the OP was experiencing had been explained and solved. That's
    what allowed the OP to post this reply:

    >It's works ! thanks (more 500*500*500*4).
    >But do you know why it don't work with static data ?

    You have digressed into a different area entirely.

    - Wayne



    • Edited by WayneAKing Friday, October 3, 2014 8:53 PM
    Friday, October 3, 2014 8:49 PM

  • Yes my problem is resolved, now it's just a question of detail of my project.

    For more detail of project, i uploading my project:

    engine




    • Edited by shaynox Friday, October 3, 2014 9:14 PM
    Friday, October 3, 2014 9:04 PM

  • Yes my problem is resolved, now it's just a question of detail of my project.

    For more detail of project, i uploading my project:

    engine




    I have that now, but its at the end of the day. I will check it out for you and get back here on Monday.



    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Friday, October 3, 2014 11:19 PM

  • Yes my problem is resolved, now it's just a question of detail of my project.

    For more detail of project, i uploading my project:

    engine




    This is built with the Intel compiler, this forum is for Visual C++ users

    Please go to the Intel forums for help with their development tools

    https://communities.intel.com/activity



    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Saturday, October 4, 2014 3:18 PM
  • It's work with microsoft compiler too, need to go to properties of project.C/C++.General [Intel C++] and change No to Yes.
    Saturday, October 4, 2014 3:53 PM
  • It's work with microsoft compiler too, need to go to properties of project.C/C++.General [Intel C++] and change No to Yes.

    It has some dependencies that I cannot provide as I do not have an Intel compiler



    MSFT Signature

    Place your rig specifics into your signature like I have, makes it 100x easier!

    Hardcore Games Legendary is the Only Way to Play!
    Vegan Advocate How can you be an environmentalist and still eat meat?

    Saturday, October 4, 2014 4:02 PM
  • ah ok, sorry
    Sunday, October 5, 2014 11:42 AM