Tuesday 26 March 2013

Memory usage

There are several tools to see the memory usage of Rebol.
The first one is to see the system/stats:

system/stats
== 5524993


The value showed represents the bytes used, you have to divide to 106 to find out how many Megabytes are exactly, so you may write:

system/stats / (10 ** 6)
== 5.525153


The second method is to use the stats function, it may show:
  • total memory used

    stats
    == 5524993

  • /pools Returns: width units free-units units-per-alloc segments mem-in-use

    >> ? stats/pools
    == STATS/POOLS is a block of value: [[16 5888 342 256 23 94392] [32 4224 1040 128 33 135432]
    [48 11520 2683 128 90 553680] [64 6208 1974 64 97 398088] [80 2368 556 64 37 189736] [96 1504 301 32 47 144760] [112 1312 119 32 41 147272] [128 768 20 32 24 98496] [144 512 118 32 16 73856] [160 480 187 32 15 76920] [176 320 77 32 10 56400] [192 384 110 32 12 73824 ] [208 224 70 32 7 46648] [224 160 42 32 5 35880] [240 160 45 32 5 38440] [256 128 34 324 32800] [512 784 177 16 49 401800] [1024 504 10 8 63 516600] [2048 64 13 8 8 131136] [40 96 304 292 4 76 1245792] [16 37376 7742 512 73 598600] [16 2304 812 256 9 36936] [1 23879 68 15 1 0 0]]

  • /types Returns: Datatype count
    >> ? stats/types
    == STATS/TYPES is a block of value: [end! 30 unset! 4130 error! 2 datatype! 1406 context! 0
    native! 151 action! 126 routine! 0 op! 19 function! 712 object! 3193 struct! 0 library! 0
    port! 20 any-type! 0 any-word! 0 any-function! 0 number! 0 series! 0 any-string! 0 any-block! 0 symbol! 2947 word! 58270 set-word! 6725 get-word! 541 lit-word! 1358 refinement! 1397 none! 9679 logic! 585 integer! 5750 decimal! 35 money! 0 time! 6 date! 48 char! 235
    pair! 1953 event! 114 tuple! 947 bitset! 42 string! 3795 issue! 10 binary! 20 file! 107 email! 9 url! 57 tag! 2 image! 55 block! 11264 paren! 712 path! 5695 set-path! 1708 lit-path! 0 hash! 1 list! 0]

  • /series Returns: total, blocks, strings, other, avail, free, expansions
    >> ? stats/series
    == STATS/SERIES is a block of value: [37376 22797 6777 23 7779 7778 949]

    (are avail and free always the same?)
  • /frames Returns: total, used, unused, free, values-total
    >> ? stats/frames
    == STATS/FRAMES is a block of value: [2304 1492 812 812 22246]

    (are unused and free always the same?)
  • /recycle Returns: count, series-total, series-last, frames-total, frames-last, ballast
  • >> ? stats/recycle == STATS/RECYCLE is a block of value: [17 16443 451 1263 32 2998576]
  • /evals Returns: values functions blocks
    >> ? stats/evals
    == STATS/EVALS is a block of value: [112536 44009 16120]

  • /clear Clears the evals counters
  • >> ? stats/clear == STATS/CLEAR is an integer of value: 5536561

Another very handle tool is the mem-stats.r script, it show a window with all memory stats and you may use the console in the same time and test your scripts:

Here is the source:
REBOL [
    Title: Name: "Memory Watcher"
    Author: "Nenad Rakocevic"
    Email: dockimbel@free.fr
    Date: 12/02/2001
    File: %mem-watch.r
    Version: 1.0.0
    Purpose: "Watch memory evolve during a console session"
    Needs: [view]
    Usage: {
        Just do :
            >> do %mem-watch.r
       
        then at any time :
            >> mem-watch
       
        Now you're in the mem-watch console (rebol-like). Type anything
        on the command-line and watch memory change !
       
        The red background shows last modified values.
       
        Type 'quit to exit from mem-watcher.
       
        You can even run complete scripts, but you'll see memory
        changes only after the end of your script.
        If you want to see changes during your script evaluation,
        just place in your script calls to the 'refresh-watcher
        function.
       
        refresh-watcher         ; update the memory window.
        refresh-watcher/wait     ; update the memory window and wait until
                                ; you press a key.
                               
        Note: a 'recycle is done each time the prompt is printed.
    }
    Comment: {
        - All the infos are taken for Carl's mem-stats.r script.
        - If you're always worrying about memory, mem-watch will show you
          where it's gone !
        - Won't work with /View apps (for now)
        - This script could much more handier if it could show also
          the difference between two updates for each values.
        - 'Throws are not catched in the mem-watch console.
        - I'm not sure that my approach is really accurate. 8/
    }
]
watcher-object: context [
    norm-color: 255.255.255
    high-color: 250.160.160
    watcher: total: plane: none
   
    set 'refresh-watcher func [/wait /local blk val row item p-face][
        ;--- Updating total mem ---
        either total/text = val: mold system/stats [
            total/color: norm-color
        ][
            total/text: :val
            total/color: high-color
        ]  
        ;--- Updating general infos ---
        blk: join join system/stats/recycle system/stats/series system/stats/frames
        p-face: plane/pane
        foreach item blk [
            either p-face/1/text = val: mold item [
                p-face/1/color: norm-color
            ][
                p-face/1/text: :val
                p-face/1/color: high-color
            ]
            p-face: next p-face
        ]
        ;--- Updating pools ---
        blk: system/stats/pools
        p-face: at plane/pane 19
        foreach row blk [
            foreach item row [
                either p-face/1/text = val: mold item [
                    p-face/1/color: norm-color
                ][
                    p-face/1/text: :val
                    p-face/1/color: high-color
                ]
                p-face: next p-face
            ]
        ]
        show watcher
        if wait [ask "Press Enter key to continue..."]
    ]
    ss: stylize [
        lab: tt 60 right black font []
            edge [size: 1x1 color: 144.144.144 effect: 'ibevel]
            with [color: norm-color]
        txt: txt no-wrap 0.0.80
            edge [size: 1x1 color: 144.144.144 effect: 'ibevel]
            with [color: 188.188.188]
        txt60: txt 60 center
        txt-large: txt 175x18
        pool-face: face 60x18
            edge [size: 1x1 color: 144.144.144 effect: 'ibevel]
            font [color: 0.0.0 shadow: none align: 'center]
            with [color: norm-color init: []]
    ]
    set 'mem-watch func [/local watch-win pos cmd err][
        unview/all
        watcher: layout [
            styles ss
            size 605x440
            plane: backdrop 188.188.188
            space 0
            at 0x0 h5 "Total memory allocated" 0.0.80
            at 175x0 total: pool-face right "0"
            origin 60x38
            txt-large "recycles since boot"
            txt-large "series recycled since boot"
            txt-large "series last recycled"
            txt-large "frames recycled since boot"
            txt-large "frames last recycled"
            txt-large "ballast remaining"
            txt-large "total series"
            txt-large "block series"
            txt-large "string series"
            txt-large "other series"
            txt-large "unused series"
            txt-large "free series (= unused)"
            txt-large "expansions performed"
            txt-large "frames"
            txt-large "frames in use"
            txt-large "frames not in use"
            txt-large "free frames (= unused)"
            txt-large "values held in frames"
            across at 240x0
            txt60 "Width" txt60 "Units" txt60 "Free" txt60 "Segment" txt60 "Units/Alloc"
            txt60 "Bytes"
            do [
                plane/pane: make block! (23 * 6) + 18
                pos: 0x0
                loop 18 [
                    append plane/pane make ss/pool-face [
                        offset: pos + 0x38
                        font: make font [align: 'right]
                    ]
                    pos/y: pos/y + 18
                ]  
                pos: 0x0
                loop 23 [
                    loop 6 [
                        append plane/pane make ss/pool-face [offset: pos + 240x20]
                        pos/x: pos/x + 60
                    ]
                    pos/x: 0
                    pos/y: pos/y + 18
                ]
                recycle
            ]
        ]
        watcher/offset: to-pair reduce [system/view/screen-face/size/x - watcher/size/x 0]
        watch-win: view/new watcher
        ;--- Rebol console emulator ---
        print "^/Entering Memory Watcher...(type 'quit to exit)"
        forever [
            refresh-watcher
            recycle
            cmd: ask "[mem-watcher]>> "
            either "quit" = trim cmd [unview/only watch-win halt ][
                either error? set/any 'err try [do cmd][
                    err: disarm err
                    print ["**" err/type "error:" err/id "on" remold [err/arg1]]
                    print ["** Where:" mold err/near]
                ][
                    if all [
                        not unset? 'err
                        value? 'err
                        not object? err
                        not port? err
                    ][
                        if all [series? :err 80 < length? :err][err: join mold copy/part err 80 "..." ]
                        print rejoin [system/console/result :err]
                    ]
                ]
                unset 'err
            ]
        ]
    ]
]
;You should comment this line (this is only usefull when running from TestPanel)
mem-watch

Another useful script is mem2.r, it gives you the opportunity to check a single function or block. This script gives you several function: mem-all, mem-do, mem-func, mem-free, pool. Let's see the some examples:

>> mem-all "[1 2 3]"
Load : 128
Execute : 128
== 0

>> mem-all "func [a b][a + b]"
Load : 224
Execute : 512
== 288

>> mem-all %test.r
>> mem-all %temp.r
Load : 2432
test done ;the script is executed
Execute : 2496
== 64

>> mem-func :help
Load : 14336
Execute : 27904
== 13568

>> print pool
3289056

>> print mem-free
2404310


Here is the source:
REBOL [
    Title: "Mem-Usage - a set of routines to test memory usage"
    Date: 9-Oct-2001
    Version: 0.0.2
    File: %mem2.r
    Author: "Romano Paolo Tenca"
    Purpose: {Functions to test memory usage}
Notes: {
pool: "Return the total of used memory in the allocated pool"
mem-free: "Return the total of free memory in the allocated pool"
mem-ld: return memory usage of "load string"
mem-tb: return memory usage of "to-block string"
mem-do: return memory usage of "do load string"
mem-all: call mem-ld and mem-do and return the difference
mem-func: memory usage of loading and creating a function
Examples:
^-a: 1
^-mem-all ""^-^-^-; memory usage of a empty block
^-mem-all "a"^-^-^-; memory usage of a block with a word
^-mem-all "func [][]"^-; memory usage of a new void func
^-mem-all "make object! []"^-; memory usage of a void object
^-mem-all %prova.r^-; memory usage of executing a rebol program
^-mem-do "help 4"     ; memory usage of executing some code
^-mem-func :help^-     ; memory usage of re-loading and re-creating a function
^-print pool^-^-^-; memory used
^-print mem-free^-^-; memory free
}
    History: [0.0.2 "First public release" ]
    Email: rotenca@libero.it
    library: [
        level: 'intermediate
        platform: []
        type: []
        domain: [debug]
        tested-under: none
        support: none
        license: none
        see-also: none
    ]
]
pool: has ["Return the total of used memory in the allocated pool"][
    tot-used: 0
    foreach y system/stats/pools [
        if 1 <> first y [
            tot-used: y/2 - y/3 * y/1 + tot-used
        ]
    ]
    tot-used
]
mem-free: has ["Return the total free memory in the allocated pool"] [system/stats - pool]
;funzione per stabilire il valore da sottrarre
init-mem: func [code /local old new ret][
    recycle
    old: pool
    new: pool
    print ["Pool usage:" ret: new - old]
    ret
]
mem-tb: func [{Return memory usage of "to-block string"} code [string! file!] /local old new ret][
    recycle
    old: pool
    to-block code
    new: pool
    print ["To block   :" ret: new - old - pool-usg]
    ret
]
mem-ld: func [{Return memory usage of "load string"} code [string! file!] /local old new ret][
    recycle
    old: pool
    load code
    new: pool
    print ["Load       :" ret: new - old - pool-usg]
    ret
]
mem-do: func [{Return memory usage of "do load string"} code [string! file!] /local old new ret][
    recycle
    old: pool
    do load code
    new: pool
    print ["Execute   :" ret: new - old - pool-usg]
    ret
]
mem-func: func [{Return difference between "load a function" and "load and create a function"} code [function!]][- (mem-ld mold :code) + (mem-do mold :code) ]
mem-all:   func [{Return difference between "load string" and "do load string"} code [string! file!]][- (mem-ld :code) + (mem-do :code) ]
;example
pool-usg: init-mem ""
mem-ld ""
mem-do ""
mem-all ""
;change 'comment in 'do to try all examples
comment [
    a: 1
    mem-all ""           ; memory usage of a empty block
    mem-all "a"         ; memory usage of a block with a word
    mem-all "func [][]" ; memory usage of a new void func
    mem-all "make object! []"   ; memory usage of a void object
    ;mem-all %prova.r   ; memory usage of a rebol program
    mem-do "help 4"     ; memory usage of using a function
    mem-func :help       ; memory usage of re-loading and re-creating a function
    print pool           ; memory used
    print mem-free       ; memory free
    recycle
    print pool           ; memory used after recycle
    print mem-free       ; memory free after recycle
]
print system/script/header/Purpose
ask "See the source for examples - Return to Quit - Esc for Shell "
                                                                                                         

The last memory tool showed today is from Carl Sassenrath (Rebol Author), it's just a function that show you all memory statistics:

>> mem-stats
REBOL MEMORY STATISTICS

>> mem-stats
REBOL MEMORY STATISTICS

-------- RECYCLE STATS:
      26 recycles since boot
   20256 series recycled since boot
      56 series last recycled
    2046 frames recycled since boot
       6 frames last recycled
 2921344 ballast remaining

-------- SERIES STATS:
   37376 total series
   24373 block series
    7793 string series
      41 other series
    5169 unused series
    5168 free series (should be same as above)
    1920 expansions performed

-------- FRAME STATS:
    2560 total frames
    1638 frames in use
     922 frames not in use
     922 free frames (should be same as above)
   19517 values held in frames

-------- MEMORY POOLS:
      16 wide     5888 units       36 free       23 segs      256 per    94392 bytes
      32 wide     5248 units     1524 free       41 segs      128 per   168264 bytes
      48 wide    11392 units     1461 free       89 segs      128 per   547528 bytes
      64 wide     6272 units     1572 free       98 segs       64 per   402192 bytes
      80 wide     2496 units      557 free       39 segs       64 per   199992 bytes
      96 wide     1792 units      436 free       56 segs       32 per   172480 bytes
     112 wide     1152 units      191 free       36 segs       32 per   129312 bytes
     128 wide      736 units       25 free       23 segs       32 per    94392 bytes
     144 wide      832 units      381 free       26 segs       32 per   120016 bytes
     160 wide      544 units      218 free       17 segs       32 per    87176 bytes
     176 wide      320 units       58 free       10 segs       32 per    56400 bytes
     192 wide      384 units       98 free       12 segs       32 per    73824 bytes
     208 wide      224 units       26 free        7 segs       32 per    46648 bytes
     224 wide      160 units       29 free        5 segs       32 per    35880 bytes
     240 wide      192 units       69 free        6 segs       32 per    46128 bytes
     256 wide      128 units       26 free        4 segs       32 per    32800 bytes
     512 wide      768 units       93 free       48 segs       16 per   393600 bytes
    1024 wide      464 units       22 free       58 segs        8 per   475600 bytes
    2048 wide       88 units       23 free       11 segs        8 per   180312 bytes
    4096 wide      312 units      291 free       78 segs        4 per  1278576 bytes
      16 wide    37376 units     5081 free       73 segs      512 per   598600 bytes
      16 wide     2560 units      921 free       10 segs      256 per    41040 bytes
       1 wide  2406400 units       26 free        0 segs        1 per        0 bytes
 5801750 bytes total memory allocated by REBOL kernel

-------- TOTAL DATATYPES:
      84 end!
    4112 unset!
       2 error!
    1418 datatype!
       0 context!
     147 native!
     126 action!
      17 routine!
      19 op!
     880 function!
    2927 object!
       2 struct!
       5 library!
      43 port!
       0 any-type!
       0 any-word!
       0 any-function!
       0 number!
       0 series!
       0 any-string!
       0 any-block!
    3020 symbol!
   64383 word!
    5834 set-word!
     567 get-word!
    1135 lit-word!
    1512 refinement!
    8181 none!
     455 logic!
    6692 integer!
      66 decimal!
       1 money!
       7 time!
      50 date!
     267 char!
    1377 pair!
       7 event!
     507 tuple!
      43 bitset!
    4622 string!
      11 issue!
      23 binary!
     296 file!
       9 email!
      85 url!
       2 tag!
     119 image!
   11401 block!
     797 paren!
    6240 path!
    1912 set-path!
       0 lit-path!
       0 hash!
       0 list!

Here is the source:
REBOL [
    Title: "REBOL Memory Stats"
    Date: 21-Jun-2000
    File: %mem-stats.r
    Author: "Carl Sassenrath"
    Purpose: {Print out statistics for memory usage. (Command only.) }
]
stats: get in system 'stats
fmt: func [n /local str] [
    str: form n
    head insert/dup str " " (8 - length? str)
]
peach: func [title vals strings] [
    print [newline "--------" title]
    foreach val vals [
        print [fmt val first strings]
        strings: next strings
    ]
]
mem-stats: does [
    print "REBOL MEMORY STATISTICS"
    peach "RECYCLE STATS:" stats/recycle [
        "recycles since boot"
        "series recycled since boot"
        "series last recycled"
        "frames recycled since boot"
        "frames last recycled"
        "ballast remaining"
    ]
    peach "SERIES STATS:" stats/series [
        "total series"
        "block series"
        "string series"
        "other series"
        "unused series"
        "free series (should be same as above)"
        "expansions performed"
    ]
    peach "FRAME STATS:" stats/frames [
        "total frames"
        "frames in use"
        "frames not in use"
        "free frames (should be same as above)"
        "values held in frames"
    ]
    print "^/-------- MEMORY POOLS:"
    foreach a stats/pools [
        print [
            fmt a/1 "wide"
            fmt a/2 "units"
            fmt a/3 "free"
            fmt a/5 "segs"
            fmt a/4 "per"
            fmt a/6 "bytes"
        ]
    ]
    print [fmt stats "bytes total memory allocated by REBOL kernel"]
    print "^/-------- TOTAL DATATYPES:"
    foreach [type cnt] stats/types [print [fmt cnt type]]
]
mem-stats
halt

No comments:

Post a Comment