Hướng dẫn python shared memory
Source code: Lib/multiprocessing/shared_memory.py New in version 3.8. This module provides a class, In this module, shared memory refers to “System V style” shared memory blocks (though is not necessarily implemented explicitly as such) and does not refer to “distributed shared memory”. This style of shared memory permits distinct processes to potentially read and write to a common (or shared) region of volatile memory. Processes are conventionally limited to only have access to their own process memory space but shared memory
permits the sharing of data between processes, avoiding the need to instead send messages between processes containing that data. Sharing data directly via memory can provide significant performance benefits compared to sharing data via disk or socket or other communications requiring the serialization/deserialization and copying of data. Creates a new shared memory block or attaches to an existing shared memory block. Each shared memory block is assigned a unique name. In this way, one process can create a shared memory block with a particular name and a different process can attach to that same shared memory block using that same name. As a resource for sharing data across processes, shared memory blocks may outlive the original process that created them.
When one process no longer needs access to a shared memory block that might still be needed by other processes, the name is the unique name for the requested shared memory, specified as a string. When creating a new shared memory block, if create
controls whether a new shared memory block is created ( size specifies the requested number of bytes when creating a new shared memory block. Because some platforms choose to allocate chunks of memory based upon that platform’s memory page size, the exact size of the shared memory block may be larger or equal to the size requested. When attaching to an existing shared memory block, the close ()¶Closes access to the shared memory from this instance. In order to ensure proper cleanup of resources, all instances should call unlink ()¶Requests that the underlying shared memory block be destroyed. In order to ensure proper cleanup of resources, buf ¶ A memoryview of contents of the shared memory block. name ¶Read-only access to the unique name of the shared memory block. size ¶Read-only access to size in bytes of the shared memory block. The following example demonstrates low-level use of >>> from multiprocessing import shared_memory >>> shm_a = shared_memory.SharedMemory(create=True, size=10) >>> type(shm_a.buf) The following example demonstrates a practical use of the >>> # In the first Python interactive shell >>> import numpy as np >>> a = np.array([1, 1, 2, 3, 5, 8]) # Start with an existing NumPy array >>> from multiprocessing import shared_memory >>> shm = shared_memory.SharedMemory(create=True, size=a.nbytes) >>> # Now create a NumPy array backed by shared memory >>> b = np.ndarray(a.shape, dtype=a.dtype, buffer=shm.buf) >>> b[:] = a[:] # Copy the original data into shared memory >>> b array([1, 1, 2, 3, 5, 8]) >>> type(b)class multiprocessing.managers. SharedMemoryManager ([address[, authkey]])¶A subclass of A call to This class provides methods for creating and returning Refer
to SharedMemory (size)¶Create and return a new ShareableList (sequence)¶Create and return a new The following
example demonstrates the basic mechanisms of a >>> from multiprocessing.managers import SharedMemoryManager >>> smm = SharedMemoryManager() >>> smm.start() # Start the process that manages the shared memory blocks >>> sl = smm.ShareableList(range(4)) >>> sl ShareableList([0, 1, 2, 3], name='psm_6572_7512') >>> raw_shm = smm.SharedMemory(size=128) >>> another_sl = smm.ShareableList('alpha') >>> another_sl ShareableList(['a', 'l', 'p', 'h', 'a'], name='psm_6572_12221') >>> smm.shutdown() # Calls unlink() on sl, raw_shm, and another_sl The following example depicts a potentially more convenient pattern for using >>> with SharedMemoryManager() as smm: ... sl = smm.ShareableList(range(2000)) ... # Divide the work among two processes, storing partial results in sl ... p1 = Process(target=do_work, args=(sl, 0, 1000)) ... p2 = Process(target=do_work, args=(sl, 1000, 2000)) ... p1.start() ... p2.start() # A multiprocessing.Pool might be more efficient ... p1.join() ... p2.join() # Wait for all work to complete in both processes ... total_result = sum(sl) # Consolidate the partial results now in sl When using a multiprocessing.shared_memory. ShareableList (sequence=None, *, name=None)¶Provides a mutable list-like object where all values stored within are stored in a shared memory block. This constrains storable values to only the sequence is used in populating a new name is the unique name for the requested shared memory, as described in the definition for
count (value)¶Returns the number of occurrences of index (value)¶Returns first index position of format ¶Read-only attribute containing the shm ¶The The following example demonstrates basic use of a >>> from multiprocessing import shared_memory >>> a = shared_memory.ShareableList(['howdy', b'HoWdY', -273.154, 100, None, True, 42]) >>> [ type(entry) for entry in a ] [ The following example depicts how one, two, or many processes may access the same >>> b = shared_memory.ShareableList(range(5)) # In a first process >>> c = shared_memory.ShareableList(name=b.shm.name) # In a second process >>> c ShareableList([0, 1, 2, 3, 4], name='...') >>> c[-1] = -999 >>> b[-1] -999 >>> b.shm.close() >>> c.shm.close() >>> c.shm.unlink() The following examples demonstrates that >>> import pickle >>> from multiprocessing import shared_memory >>> sl = shared_memory.ShareableList(range(10)) >>> list(sl) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >>> deserialized_sl = pickle.loads(pickle.dumps(sl)) >>> list(deserialized_sl) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >>> sl[0] = -1 >>> deserialized_sl[1] = -2 >>> list(sl) [-1, -2, 2, 3, 4, 5, 6, 7, 8, 9] >>> list(deserialized_sl) [-1, -2, 2, 3, 4, 5, 6, 7, 8, 9] >>> sl.shm.close() >>> sl.shm.unlink() |