Hello! again esenthel, i last came to you with a problem regarding heightmaps and my processor, well this time it's to do with the uploader tool! When i try to upload a 1.2 GB pak file the error i get is can't allocate 123 335 345 bytes of memory.
And this is only on my computer, the memory gets out of control going up to a whopping 3.4 GB then crashing on a 1.2 gb file. Same with a 700 MB file. it just wont upload.
We tried the same settings on my team mates computer, it started uploading instantly and there were no memory leaks/errors.
Computer not overclocked at all.
Error and memory being used:
http://puu.sh/1A7xu

Hi,

Why are you trying to upload 1.2GB or 700MB files?
Are these pak files?
You should not upload paks, it is recommended to extract the files from paks, so they are uploaded separately. This way you don't force the user to redownload GB of data, when you change a single file inside a pak.

Uploader uses LZMA compression with highest compression level, which by itself may use hundreds of MB RAM just for compression.
Then uploader needs to have both original file and compressed file in the memory, which can be 1GB x2 if you have 1GB pak file.

Please extract the paks, and then upload single files.
Most likely the problem will go away.

I was using pak files.


Also do you have any comments on my raycasting thread?, thanks.

Your program will be downloading files individually, but when using secure, they will be encrypted, so there's no worry about stealing game files.

(12-12-2012 06:19 PM)Esenthel Wrote:  Your program will be downloading files individually, but when using secure, they will be encrypted, so there's no worry about stealing game files.

thank you very much, puts my fears at rest.

to clarify more, files on the server will be stored compressed and encrypted, after downloading them, the patcher class will automatically decrypt them in memory, so when writing together multiple files into paks on disk you need to use pak encryption (for PakCreate/PakUpdate)
please check patcher tutorial
if you expect to download few GB of data at one time, then it's recommended to create a few paks, each sized approximately 500MB