Python ´òËãɾ³ý´óÁ¿Éæ¼°ÏñCºÍC++ÓïÑÔÄÇÑùµÄ¸´ÔÓÄÚ´æ¹ÜÀí¡£µ±¶ÔÏóÀ뿪·¶Î§£¬¾Í»á±»×Ô¶¯À¬»øÊÕ¼¯Æ÷»ØÊÕ¡£È»¶ø£¬¶ÔÓÚÓÉ Python ¿ª·¢µÄ´óÐÍÇÒ³¤ÆÚÔËÐеÄϵͳÀ´Ëµ£¬ÄÚ´æ¹ÜÀíÊDz»ÈÝСêïµÄÊÂÇé¡£

ÔÚÕâƪ²©¿ÍÖУ¬ÎÒ½«»á·ÖÏí¹ØÓÚ¼õÉÙ Python ÄÚ´æÏûºÄµÄ·½·¨ºÍ·ÖÎöµ¼ÖÂÄÚ´æÏûºÄ/ÅòÕ͸ùÔ´µÄÎÊÌâ¡£ÕâЩ¶¼ÊÇ´Óʵ¼Ê²Ù×÷ÖÐ×ܽáµÄ¾­Ñ飬ÎÒÃÇÕýÔÚ¹¹½¨ Datos IO µÄ RecoverX ·Ö²¼Ê½±¸·ÝºÍ»Ö¸´Æ½Ì¨£¬ÕâÀïÖ÷ÒªÒª½éÉܵÄÊÇÔÚ Python£¨ÔÚ C++ £¬Java ºÍ bash ÖÐÒ²ÓÐһЩÀàËƵÄ×é¼þ£© ÖеĿª·¢¡£

Python À¬»øÊÕ¼¯

Python½âÊÍÆ÷¶ÔÕýÔÚʹÓõĶÔÏó±£³Ö¼ÆÊý¡£µ±¶ÔÏó²»ÔÙ±»ÒýÓÃÖ¸ÏòµÄʱºò£¬À¬»øÊÕ¼¯Æ÷¿ÉÒÔÊͷŸöÔÏ󣬻ñÈ¡·ÖÅäµÄÄÚ´æ¡£ÀýÈ磬Èç¹ûÄãʹÓó£¹æµÄPython(CPython, ²»ÊÇJPython)ʱ£¬PythonµÄÀ¬»øÊÕ¼¯Æ÷½«µ÷ÓÃfree()/delete() ¡£

ʵÓù¤¾ß

×ÊÔ´£¨resource£©

resource Ä£¿éÓÃÀ´²é¿´ÏîÄ¿µ±Ç°µÃµÄ¹ÌÓеÄ)ÄÚ´æÏûºÄ(¹ÌÓÐÄÚ´æÊÇÏîĿʵ¼ÊʹÓõÄRAM)£¬×¢Òâresource¿âÖ»ÔÚlinuxϵͳÏÂÓÐЧ

>>> import resource
>>> resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
443

¶ÔÏó£¨objgraph£©

objgraph ÊÇÒ»¸öʵÓÃÄ£¿é,¿ÉÒÔչʾµ±Ç°ÄÚ´æÖдæÔڵĶÔÏó

À´¿´¿´objgraphµÄ¼òµ¥Ó÷¨£º

import objgraph
import random
import inspect

class Foo(object):

  def __init__(self):
    self.val = None

  def __str__(self):
    return "foo - val: {0}".format(self.val)

def f():

  l = []

  for i in range(3):
    foo = Foo()
    l.append(foo)

  return l


def main():

  d = {}

  l = f()

  d['k'] = l

  print "list l has {0} objectsoftype Foo()".format(len(l))

pythontest1.py

Êä³ö£º

list l has 10000 objectsoftype Foo()
dict 10423
Foo 10000 ¡ª¡ª¡ª¡ª> Guiltyas charged!
tuple 3349
wrapper_descriptor 945
function 860
builtin_function_or_method 616
method_descriptor 338
weakref 199
member_descriptor 161
getset_descriptor 107

×¢Ò⣬ÎÒÃÇÔÚÄÚ´æÖл¹³ÖÓÐ10,423¸ö¡®dict'µÄʵÀý¶ÔÏó¡£

¿ÉÊÓ»¯objgraphÒÀÀµÏî

ObjgraphÓиö²»´íµÄ¹¦ÄÜ£¬¿ÉÒÔÏÔʾFoo()¶ÔÏóÔÚÄÚ´æÖдæÔÚµÄÒòËØ£¬¼´£¬ÏÔʾ˭³ÖÓжÔËüµÄÒýÓà (ÔÚÕâ¸öÀý×ÓÖÐÊÇlist l)¡£

ÔÚRedHat/CentosÉÏ, Äã¿ÉÒÔʹÓÃsudo yum install graphviz*°²×°graphviz
ÔÚUbunbuµÈϵͳÉÏʹÓÃsudo apt-get install graphviz*°²×°graphviz

ÈçÐè²é¿´¶ÔÏó×Öµä d£¬Çë²Î¿¼:

objgraph.show_refs(d, filename='sample-graph.png')

´ÓÄÚ´æʹÓýǶÈÀ´¿´£¬ÎÒÃǾªÆæµØ·¢ÏÖ¡ª¡ªÎªÊ²Ã´¶ÔÏóûÓÐÊÍ·Å?ÕâÊÇÒòΪÓÐÈËÔÚ³ÖÓжÔËüµÄÒýÓá£

Õâ¸öСƬ¶ÎչʾÁËobjgraphÔõÑùÌṩÏà¹ØÐÅÏ¢:

objgraph.show_backrefs(random.choice(objgraph.by_type('Foo')), filename="foo_refs.png")

ÔÚÕâÒ»°¸ÀýÖÐ, ÎÒÃDz鿴ÁËFooÀàÐ͵ÄËæ»ú¶ÔÏó¡£ÎÒÃÇÖªµÀ¸ÃÌض¨¶ÔÏó±»±£´æÔÚÄÚ´æÖУ¬ÒòÆäÒýÓÃÁ´½ÓÔÚÖ¸¶¨·¶Î§ÄÚ¡£

ÓÐʱ£¬ÒÔÉϼ¼ÇÉÄÜ°ïÖúÎÒÃÇÀí½â£¬ÎªÊ²Ã´µ±ÎÒÃDz»ÔÙʹÓÃij¶ÔÏóʱ£¬PythonÀ¬»ø»ØÊÕÆ÷ûÓн«À¬»ø»ØÊÕ¡£

ÄÑ´¦ÀíµÄÊÇ£¬ÓÐʱºòÎÒÃǻᷢÏÖFoo()Õ¼ÓÃÁ˺ܶàÄÚ´æµÄÀà¡£ÕâʱÎÒÃÇ¿ÉÒÔÓÃheapy()À´»Ø´ðÒÔÉÏÎÊÌâ¡£

Heapy

heapy ÊÇÒ»¸öʵÓõģ¬ÓÃÓÚµ÷ÊÔÄÚ´æÏûºÄ/й©µÄ¹¤¾ß¡£Í¨³££¬ÎÒ½«objgraphºÍheapy´îÅäʹÓãºÓà heapy ²é¿´·ÖÅä¶ÔÏóËæʱ¼äÔö³¤µÄ²îÒ죬heapyÄܹ»ÏÔʾ¶ÔÏó³ÖÓеÄ×î´óÄÚ´æµÈ£»ÓÃObjgraphÕÒbackrefÁ´£¨ÀýÈ磺ǰ4½Ú£©£¬³¢ÊÔ»ñÈ¡ËüÃDz»Äܱ»ÊͷŵÄÔ­Òò¡£

HeapyµÄµäÐÍÓ÷¨ÊÇÔÚ²»Í¬µØ·½µÄ´úÂëÖе÷ÓÃÒ»¸öº¯Êý£¬ÊÔͼΪÄÚ´æʹÓÃÁ¿Ìṩ´óÁ¿ÊÕ¼¯ÏßË÷£¬ÕÒµ½¿ÉÄÜ»áÒý·¢µÄÎÊÌ⣺

from guppyimport hpy


def dump_heap(h, i):
  """
  @param h: Theheap (from hp = hpy(), h = hp.heap())
  @param i: Identifierstr
  """

  print "Dumpingstatsat: {0}".format(i)

  print 'Memoryusage: {0}(MB)'.format(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss/1024)

  print "Mostcommontypes:"
  objgraph.show_most_common_types()

  print "heapis:"
  print "{0}".format(h)

  by_refs = h.byrcs
  print "byreferences: {0}".format(by_refs)
  print "Morestatsfor topelement.."
  print "Byclodo (class or dict owner): {0}".format(by_refs[0].byclodo)
  print "Bysize: {0}".format(by_refs[0].bysize)
  print "Byid: {0}".format(by_refs[0].byid)

¼õÉÙÄÚ´æÏûºÄС¼¼ÇÉ

ÔÚÕâÒ»²¿·Ö£¬ÎÒ»á½éÉÜһЩ×Ô¼º·¢ÏֵĿɼõÉÙÄÚ´æÏûºÄµÄСÇÏÃÅ.

Slots

µ±ÄãÓÐÐí¶à¶ÔÏóʱºò¿ÉÒÔʹÓÃSlots¡£Slotting´«´ï¸øPython½âÊÍÆ÷£ºÄãµÄ¶ÔÏó²»ÐèÒª¶¯Ì¬µÄ×Öµä(´ÓÉÏÃæµÄÀý×Ó2.2ÖУ¬ÎÒÃÇ¿´µ½Ã¿¸öFoo()¶ÔÏóÄÚ²¿°üº¬Ò»¸ö×Öµä)

ÓÃslots¶¨ÒåÄãµÄ¶ÔÏó£¬ÈÃpython½âÊÍÆ÷ÖªµÀÄãµÄÀàÊôÐÔ/³ÉÔ±Êǹ̶¨µÄ.¡£ÕâÑù¿ÉÒÔÓÐЧµØ½ÚÔ¼ÄÚ´æ!

²Î¿¼ÒÔÏ´úÂ룺

import resource

class Foo(object):
  #__slots__ = ('val1', 'val2', 'val3', 'val4', 'val5', 'val6')

  def __init__(self, val):
    self.val1 = val+1
    self.val2 = val+2
    self.val3 = val+3
    self.val4 = val+4
    self.val5 = val+5
    self.val6 = val+6

def f(count):
  l = []
  for i in range(count):
    foo = Foo(i)
    l.append(foo)

  return l

def main():
  count = 10000
  l = f(count)

  mem = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss

  print "Memoryusageis: {0} KB¡±.format(mem)

  print "Sizeperfooobj: {0} KB¡±.format(float(mem)/count)

if __name__ == "__main__¡±:
  main()

[vagrant@datosdevtemp]$ pythontest2.py

Êä³ö£º

Memoryusageis: 16672 KB
Sizeperfooobj: 1.6672 KB
Nowun-commentthisline: #__slots__ = (¡®val1', ¡®val2', ¡®val3', ¡®val4', ¡®val5', ¡®val6')
[vagrant@datosdevtemp]$ pythontest2.py
Memoryusageis: 6576 KB
Sizeperfooobj: 0.6576 KB

ÔÚÕâ¸öÀý×ÓÖУ¬¼õÉÙÁË60%µÄÄÚ´æÏûºÄ£¡

פÁô£º½÷·ÀפÁô×Ö·û´®£¡

Python»á¼Ç¼Èç×Ö·û´®µÈ²»¿É¸Ä±äµÄÖµ£¨Æäÿ¸öÖµµÄ´óСÒÀÀµÓÚʵÏÖ·½·¨£©£¬Õâ³ÆΪפÁô¡£

>>> t = "abcdefghijklmnopqrstuvwxyz"
>>>> p = "abcdefghijklmnopqrstuvwxyz"
>>>> id(t)
139863272322872
>>> id(p)
139863272322872

ÕâÊÇÓÉpython½âÎöÆ÷Íê³ÉµÄ£¬ÕâÑù×ö¿ÉÒÔ½ÚÊ¡Äڴ棬²¢¼Ó¿ì±È½ÏËٶȡ£ÀýÈ磬Èç¹ûÁ½¸ö×Ö·û´®ÓµÓÐÏàͬµÄID»òÒýÓèCËûÃǾÍÊÇÈ«µÈµÄ¡£

È»¶ø£¬Èç¹ûÄãµÄ³ÌÐò´´½¨ÁËÐí¶àСµÄ×Ö·û´®£¬ÄãµÄÄÚ´æ¾Í»á³öÏÖÅòÕÍ¡£

Éú³É×Ö·û´®Ê±Ê¹ÓÃFormatÀ´´úÌæ¡°+¡±

½ÓÏÂÀ´£¬ÔÚ¹¹Ôì×Ö·û´®Ê±£¬Ê¹ÓÃFormatÀ´´úÌæ¡°+¡±¹¹½¨×Ö·û´®¡£

Ò༴£¬

st = "{0}_{1}_{2}_{3}".format(a,b,c,d) # ¶ÔÄÚ´æ¸üºÃ£¬²»´´½¨ÁÙʱ±äÁ¿
st2 = a + '_' + b + '_' + c + '_' + d # ÔÚÿ¸ö"+"ʱ´´½¨Ò»¸öÁÙʱstr£¬ÕâЩ¶¼ÊÇפÁôÔÚÄÚ´æÖеġ£

ÔÚÎÒÃǵÄϵͳÖУ¬µ±ÎÒÃǽ«Ä³Ð©×Ö·û´®¹¹Ôì´Ó¡°+¡±±äΪʹÓÃformatʱ£¬ÄÚ´æ»áÃ÷ÏÔ±»½ÚÊ¡¡£

¹ØÓÚϵͳ¼¶±ð

ÉÏÃæÎÒÃÇÌÖÂ۵ļ¼ÇÉ¿ÉÒÔ°ïÖúÄãÕÒ³öϵͳÄÚ´æÏûºÄµÄÎÊÌâ¡£µ«ÊÇ£¬Ëæ×Åʱ¼äµÄÍÆÒÆ£¬python½ø³Ì²úÉúµÄÄÚ´æÏûºÄ»á³ÖÐøÔö¼Ó¡£ÕâËƺõÓëÒÔÏÂÎÊÌâÓйØ:

1. ΪʲôCÖÐÄÚ´æ·ÖÅäÄܹ»ÔÚPythonÄÚ²¿Æð×÷Óã¬Õâ±¾ÖÊÉÏÊÇÄÚ´æËéƬµ¼Öµġ£ÒòΪ£¬³ý·ÇÕû¸öÄÚ´æûÓÐʹÓùý£¬·ñÔò¸Ã·ÖÅä¹ý³Ì²»Äܵ÷Óá®free'·½·¨¡£µ«ÐèҪעÒâµÄÊÇ£¬ÄÚ´æµÄʹÓò»ÊǸù¾ÝÄãËù´´½¨ºÍʹÓõĶÔÏóÀ´½øÐÐÅÅÁС£
2. ÄÚ´æÔö¼ÓÒ²ºÍÉÏÃæÌÖÂ۵ġ°Interning¡± Óйء£

ÒÔÎҵľ­ÑéÀ´¿´£¬¼õÉÙpythonÖÐÄÚ´æÏûºÄµÄ±ÈÀýÊÇ¿ÉÐеġ£ÔÚDatos IOÖУ¬ÎÒÔø¾­Õë¶ÔÖ¸¶¨µÄÄÚ´æÏûºÄ½ø³ÌʵÏÖ¹ýÒ»¸ö¹¤×÷Ä£¿é¡£¶ÔÓÚÐòÁл¯µÄ¹¤×÷µ¥Ôª£¬ÎÒÃÇÔËÐÐÁËÒ»¸ö¹¤×÷½ø³Ì¡£µ±¹¤×÷½ø³ÌÍê³Éºó, Ëü»á±»ÒƳýÁË¡ª¡ªÕâÊÇ·µ»Øϵͳȫ²¿ÄÚ´æµÄΨһ¿ÉÒÔÓÐЧ·½·¨ :)¡£ºÃµÄÄÚ´æ¹ÜÀíÔÊÐíÔö¼Ó·ÖÅäÄÚ´æµÄ´óС£¬¼´ÔÊÐí¹¤×÷½ø³Ì³¤Ê±¼äÔËÐС£

×ܽá

ÎÒ¹éÄÉÁËһЩ¼õÉÙpython½ø³ÌÏûºÄÄÚ´æµÄ¼¼ÇÉ£¬µ±ÎÒÃÇÔÚ´úÂëÖÐÑ°ÕÒÄÚ´æй©ʱ£¬Ò»ÖÖ·½·¨ÊÇͨ¹ýʹÓÃHeapyÕÒ³öÄÄЩObjÕ¼ÓÃÁ˽϶àÄڴ棬Ȼºóͨ¹ýʹÓÃObjgraphÕÒ³öÄÚ´æ±»ÊͷŵÄÔ­Òò£¨³ý·ÇÄãÈÏΪËûÃDZ¾Ó¦¸Ã±»ÊÍ·Å£©¡£

×ܵÄÀ´Ëµ£¬ÎÒ¾õµÃÔÚpythonÖÐÑ°ÕÒÄÚ´æÎÊÌâÊÇÒ»ÖÖÐÞÐС£Ëæ×Åʱ¼äµÄ»ýÀÛ£¬¶ÔÓÚϵͳÖеÄÄÚ´æÅòÕͺÍй©ÎÊÌ⣬ÄãÄܲúÉúÒ»ÖÖÖ±¾õÅжϣ¬²¢Äܸü¿ìµØ½â¾öËüÃÇ¡£Ô¸ÄãÔÚ·¢ÏÖÎÊÌâµÄ¹ý³ÌÖÐÕÒµ½ÀÖȤ£¡

ÒÔÉϾÍÊDZ¾ÎĵÄÈ«²¿ÄÚÈÝ£¬Ï£Íû¶Ô´ó¼ÒµÄѧϰÓÐËù°ïÖú£¬Ò²Ï£Íû´ó¼Ò¶à¶àÖ§³Ö½Å±¾Ö®¼Ò¡£