ram

Java's RAM usage doesn't correspond to what the Task Manager says

给你一囗甜甜゛ 提交于 2019-12-09 17:54:04
问题 I have been playing with Java's JVM by making a 1024^3 (basically 1Gb) length byte array. I measured the RAM usage before, after the array creation and after the array's destruction by the garbage collector, using both the Task Manager (looking at the process) and this little snippet: public static void showMemory() { System.out.println("Memory used: " + (Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) / (1024.D * 1024.D) + "mB."); } The aforementioned code shows 2Mb,

How to transform a huge CSV into SQLite using Pandas?

我只是一个虾纸丫 提交于 2019-12-09 13:05:16
问题 I have a huge table (about 60 GB) in form of an archived CSV file. I want to transform it into an SQLite file. What I do at the moment in the following: import pandas import sqlite3 cnx = sqlite3.connect('db.sqlite') df = pandas.read_csv('db.gz', compression='gzip') df.to_sql('table_name', cnx) It works fine for smaller files but with the huge files I have memory problem. The problem is that pandas reads the whole table into memory (RAM) and then saves it into SQLite file. Is there an elegant

Will multi threading provide any performance boost?

*爱你&永不变心* 提交于 2019-12-09 04:39:11
问题 I am new to programming in general so please keep that in mind when you answer my question. I have a program that takes a large 3D array (1 billion elements) and sums up elements along the various axis to produce a 2D array of a projection of each side of the data. The problem here is that it is very ram intensive as the program is constantly fetching information from the ram, both reading and writing. The question is, will i gain any performance increases if i multithread the program or will

index on url or hashing considering RAM

送分小仙女□ 提交于 2019-12-09 01:04:35
问题 I am working on a project which needs to add/update around 1 million urls daily. Some days are mostly updates and some days are mostly add and some days are mix. So, on every query there is need to look up uniqueness of url in url table. How look up for url can be made really fast because at the moment index is set at url column and it works good but in coming weeks RAM would not be enough if index are kept on same column and new records will be added in millions. That's why I am looking for

Why does RAM access any memory address in O(1) time?

回眸只為那壹抹淺笑 提交于 2019-12-08 12:20:27
问题 When we work with arrays, for example, assign a[10000] = 3, what it does is it first add 10000 * sizeof(int) to memory address of a, and then access that memory address. The question is, why does accessing such memory address takes only O(1) time? Basically, what's the magic behind random access memory such that it takes the same time accessing address #1 vs. address #1,000,000? 回答1: Random access memory is by definition memory that can provide the contents at any address in a constant amount

AVR XYZ Registers

ぐ巨炮叔叔 提交于 2019-12-08 10:42:45
问题 What is the difference beatwean X, Y and Z registers in AVR microcontrollers. What for each of them suitable in C compilers? And where do compilers store heap pointer, stack pointer, frame pointer? Do this registers have the same capabilities or providing addressing in different spaces(ex. EEPROM, RAM). 回答1: X Y and Z registers are actually pairs of r27:r26, r29:r28 and r31:r30 registers. Each of them can be used as indirect pointers to SRAM: ld r16, X with post-increment, or pre-decrement:

How GUI screen transition works in qml

匆匆过客 提交于 2019-12-07 15:50:00
问题 I'm a C++ developer, now studying about GUI development using QML in QtQuick. In GUI creation, only one screen is visible to the user. And based on user interaction, the screens are switched. But what actually happens behind? There are lot of info only on how to design a single screen, but very less resource for how to manage the transitions of their states. Are all the screens and components loaded when starting the application and change the layer order to display once screen, OR after an

Reading large files into R

二次信任 提交于 2019-12-07 08:12:50
问题 I am a newbie to R, but I am aware that it chokes on "big" files. I am trying to read a 200MB data file. I have tried it in csv format and also converting it to tab delimited txt but in both cases I use up my 4GB of RAM before the file loads. Is it normal that R would use 4GB or memory to load a 200MB file, or could there be something wrong with the file and it is causing R to keep reading a bunch of nothingness in addition to the data? 回答1: From ?read.table Less memory will be used if

virtual v. physical memory in assessing C/C++ memory leak

故事扮演 提交于 2019-12-07 07:30:45
问题 I have a C++ application that I am trying to iron the memory leaks out of and I realized I don't fully understand the difference between virtual and physical memory. Results from top (so 16.8g = virtual, 111m = physical): 4406 um 20 0 16.8g 111m 4928 S 64.7 22.8 36:53.65 client My process holds 500 connections, one for each user, and at these numbers it means there is about 30 MB of virtual overhead for each user. Without going into the details of my application, the only way this could sound

Load a MySQL innodb database into memory

社会主义新天地 提交于 2019-12-06 22:32:46
问题 I have a MySQL innodb database at 1.9GB, showed by following command. SELECT table_schema "Data Base Name" , sum( data_length + index_length ) / 1 048 576 as "Data Base Size in MB" , sum( data_free )/ 1 048 576 as "Free Space in MB" FROM information_schema.TABLES GROUP BY table_schema ; +--------------------+----------------------+------------------+ | Data Base Name | Data Base Size in MB | Free Space in MB | +--------------------+----------------------+------------------+ | database_name |