memory

Why does pages allocation with order of 10 or 11 using __get_free_pages() usually fail?

不打扰是莪最后的温柔 提交于 2020-07-05 03:16:16
问题 My system memory is plenty (a server with 24GB). In my system, the kernel space is allocated with 320MB and 120MB for crash kernel. The rest of the memory is used for other purposes. However, when I use __get_free_pages() to allocate contiguous pages with order of 11, and the kernel fails to allocate 2^10 pages. Why? According to makelinux The maximum allowed value for order is 10 or 11 (corresponding to 1024 or 2048 pages), depending on the architecture. The chances of an order-10 allocation

Pandas and Multiprocessing Memory Management: Splitting a DataFrame into Multiple Chunks

删除回忆录丶 提交于 2020-07-04 09:15:47
问题 I have to process a huge pandas.DataFrame (several tens of GB) on a row by row bases, where each row operation is quite lengthy (a couple of tens of milliseconds). So I had the idea to split up the frame into chunks and process each chunk in parallel using multiprocessing . This does speed-up the task, but the memory consumption is a nightmare. Although each child process should in principle only consume a tiny chunk of the data, it needs (almost) as much memory as the original parent process

Pandas and Multiprocessing Memory Management: Splitting a DataFrame into Multiple Chunks

扶醉桌前 提交于 2020-07-04 09:15:35
问题 I have to process a huge pandas.DataFrame (several tens of GB) on a row by row bases, where each row operation is quite lengthy (a couple of tens of milliseconds). So I had the idea to split up the frame into chunks and process each chunk in parallel using multiprocessing . This does speed-up the task, but the memory consumption is a nightmare. Although each child process should in principle only consume a tiny chunk of the data, it needs (almost) as much memory as the original parent process

Lisp - write to file using low memory footprint

我怕爱的太早我们不能终老 提交于 2020-07-03 09:37:44
问题 I have large hash tables that I am writing to disk as an occasional backup. I am finding that as I map the hash tables and write to a file, the RAM usage skyrockets compared to the size of the hash. I am running lisp on emacs with slime and sbcl 2.0.3.176. System is Ubuntu 19.10 on a dell server. Data is multiple levels of hash tables. The basic structure of it is: customer-ht - hash table of structs called customer, keyed on lists of integers, like (1 2), (1 3) (defstruct customer (var1 0)

double pointer vs pointer to array, incompatible pointer type

人盡茶涼 提交于 2020-07-03 07:25:34
问题 Having this: #define _DEFAULT_SOURCE 1 #include <stdio.h> #include <string.h> int main(){ char *token, org[] = "Cats,Dogs,Mice,,,Dwarves,Elves:High,Elves:Wood"; while((token=strsep(&org,","))) printf("Token: %s\n",token); } gives err (incompatible pointer type): /usr/include/string.h:439:14: note: expected ‘char ** restrict’ but argument is of type ‘char (*)[47]’ extern char *strsep (char **__restrict __stringp, I know it is different type (one has memory initialized -> org[] , but the

How to use ReadProcessMemory

柔情痞子 提交于 2020-07-03 05:07:07
问题 So I have been trying to read variables with ReadProcessMemory and finding the adresses in cheat engine worked perfectly, but as soon as I got to programming I encountered some problems. I searched for the ammo and health addresses in cheat engine and the health was a one level pointer and the ammo was a three level pointer. I tried reading the health, but everytime I read it, it returns 0. namespace AssaultCubeTrainer { public partial class MainWindow : Window { [DllImport("kernel32.dll",

What is the addressability given number of 16 address wires (bus) and 8-bit word size?

爱⌒轻易说出口 提交于 2020-06-29 10:41:34
问题 A computer has 16 address wires (address bus?) and 8-bit word size. What is the addressability? I figured out that address space is 2^16 = 65536, but I still don't know how to calculate addressability. I know addressability is the bytes each space occupies but how do I figure this out? Any help would be appreciated, especially some general formula associating word size/address bus with addressability. I'm sorry if this question is super simple. 回答1: Since there are 2^16 unique values you can

why OS create the virtual memory for running process

这一生的挚爱 提交于 2020-06-29 03:48:14
问题 when a program started the OS will create a virtual memory , which divided into stack , heap , data , text to run a process on it.I know that each segment is used for specification purpose such as text saves the binary code of program, data saves static and global variable. My question is why the OS need to create the virtual memory and divide it into the segments ? How about if OS just use the physical memory and the process run directly on the physical memory. I think maybe the answer is

How can I obtain a global mutex?

戏子无情 提交于 2020-06-28 09:21:29
问题 I'm trying to generate shared memory in a distributed manner; I also need the solution to be cross-platform. In order to do so, I need to synchronise the initialisation of this shared memory - using a mutex. This mutex, thus, needs to be shared. I've taken a look at this question: Interprocess reader/writer lock with Boost However, it suggests placing the mutex in shared memory, generated by a central process. This is the exact opposite of what I'm wanting to do. If I had this central process

Is it possible to crash this program somehow?

▼魔方 西西 提交于 2020-06-27 16:56:32
问题 I am currently learning C in one of my courses at the university. Now we have a task to solve and I am not sure how to do it. The task looks like this: "Is it possible to let this program crash with user inputs? If so, explain this case." The program we have been given is quite simple and looks like this: #include <stdio.h> // Include to use printf, scanf int main() { // Define buffers to store user name and password char username[16]; char password[16]; // Read username and password from