lib/benchmark.rb
DEFINITIONS
This source file includes following functions.
1 #
2 # benchmark.rb
3 #
4 =begin
5 2002-04-25: bmbm(): killed unused parameter @fmtstr (gotoken)
6 2001-11-26: Time.times renamed Process.times for ruby17 (gotoken#notwork.org)
7 2001-01-12: made bmbm module func. bmbm return Tms array.
8 2001-01-10: added bmbm, Job and INSTALL.rb (gotoken#notwork.org)
9 2000-04-00: report() prints tag before eval block (gotoken#notwork.org)
10 2000-02-22: report(): measure -> Benchmark::measure (nakahiro#sarion.co.jp)
11 2000-01-02: bug fix, documentation (gotoken#notwork.org)
12 2000-01-01: measure can take a tag as opt. (nobu.nakada#nifty.ne.jp)
13 2000-01-01: first release (gotoken#notwork.org)
14 =end
15
16 =begin
17 = benchmark.rb
18
19 == NAME
20 ((*benchmark.rb*)) - a benchmark utility
21
22 == SYNOPSIS
23 ----------
24 require "benchmark"
25 include Benchmark
26 ----------
27
28 == DESCRIPTION
29
30 benchmark.rb provides some utilities to measure and report the
31 times used and passed to execute.
32
33 == SIMPLE EXAMPLE
34
35 === EXAMPLE 0
36 To ((<measure>)) the times to make (({"a"*1_000_000})):
37
38 ----------
39 puts measure{ "a"*1_000_000 }
40 ----------
41
42 On my machine (FreeBSD 3.2 on P5100MHz) this reported as follows:
43
44 ----------
45 1.166667 0.050000 1.216667 ( 0.571355)
46 ----------
47
48 The above shows user time, system time, user+system, and really passed
49 time. The unit of time is second.
50
51 === EXAMPLE 1
52 To do some experiments sequentially, ((<bm>)) is useful:
53
54 ----------
55 n = 50000
56 bm do |x|
57 x.report{for i in 1..n; a = "1"; end}
58 x.report{n.times do ; a = "1"; end}
59 x.report{1.upto(n) do ; a = "1"; end}
60 end
61 ----------
62
63 The result:
64 ----------
65 user system total real
66 1.033333 0.016667 1.016667 ( 0.492106)
67 1.483333 0.000000 1.483333 ( 0.694605)
68 1.516667 0.000000 1.516667 ( 0.711077)
69 ----------
70
71 === EXAMPLE 2
72 To put a label in each ((<report>)):
73
74 ----------
75 n = 50000
76 bm(7) do |x|
77 x.report("for:") {for i in 1..n; a = "1"; end}
78 x.report("times:") {n.times do ; a = "1"; end}
79 x.report("upto:") {1.upto(n) do ; a = "1"; end}
80 end
81 ----------
82
83 The option (({7})) specifies the offset of each report accoding to the
84 longest label.
85
86 This reports as follows:
87
88 ----------
89 user system total real
90 for: 1.050000 0.000000 1.050000 ( 0.503462)
91 times: 1.533333 0.016667 1.550000 ( 0.735473)
92 upto: 1.500000 0.016667 1.516667 ( 0.711239)
93 ----------
94
95 === EXAMPLE 3
96
97 By the way, benchmarks might seem to depend on the order of items. It
98 is caused by the cost of memory allocation and the garbage collection.
99 To prevent this boresome, Benchmark::((<bmbm>)) is provided, e.g., to
100 compare ways for sort array of strings:
101
102 ----------
103 require "rbconfig"
104 include Config
105 def file
106 open("%s/lib/ruby/%s.%s/tk.rb" %
107 [CONFIG['prefix'],CONFIG['MAJOR'],CONFIG['MINOR']]).read
108 end
109
110 n = 10
111 bmbm do |x|
112 x.report("destructive!"){
113 t = (file*n).to_a; t.each{|line| line.upcase!}; t.sort!
114 }
115 x.report("method chain"){
116 t = (file*n).to_a.collect{|line| line.upcase}.sort
117 }
118 end
119 ----------
120
121 This reports:
122
123 ----------
124 Rehearsal ------------------------------------------------
125 destructive! 2.664062 0.070312 2.734375 ( 2.783401)
126 method chain 5.257812 0.156250 5.414062 ( 5.736088)
127 --------------------------------------- total: 8.148438sec
128
129 user system total real
130 destructive! 2.359375 0.007812 2.367188 ( 2.381015)
131 method chain 3.046875 0.023438 3.070312 ( 3.085816)
132 ----------
133
134 === EXAMPLE 4
135 To report statistics of sequential experiments with unique label,
136 ((<benchmark>)) is available:
137
138 ----------
139 n = 50000
140 benchmark(" "*7 + CAPTION, 7, FMTSTR, ">total:", ">avg:") do |x|
141 tf = x.report("for:") {for i in 1..n; a = "1"; end}
142 tt = x.report("times:"){n.times do ; a = "1"; end}
143 tu = x.report("upto:") {1.upto(n) do ; a = "1"; end}
144 [tf+tt+tu, (tf+tt+tu)/3]
145 end
146 ----------
147
148 The result:
149
150 ----------
151 user system total real
152 for: 1.016667 0.016667 1.033333 ( 0.485749)
153 times: 1.450000 0.016667 1.466667 ( 0.681367)
154 upto: 1.533333 0.000000 1.533333 ( 0.722166)
155 >total: 4.000000 0.033333 4.033333 ( 1.889282)
156 >avg: 1.333333 0.011111 1.344444 ( 0.629761)
157 ----------
158
159 == Benchmark module
160
161 === CONSTANT
162 :CAPTION
163 CAPTION is a caption string which is used in Benchmark::((<benchmark>)) and
164 Benchmark::Report#((<report>)).
165 :FMTSTR
166 FMTSTR is a format string which is used in Benchmark::((<benchmark>)) and
167 Benchmark::Report#((<report>)). See also Benchmark::Tms#((<format>)).
168 :BENCHMARK_VERSION
169 BENCHMARK_VERSION is version string which statnds for the last modification
170 date (YYYY-MM-DD).
171
172 === INNER CLASS
173 * ((<Benchmark::Job>))
174 * ((<Benchmark::Report>))
175 * ((<Benchmark::Tms>))
176
177 === MODULE FUNCTION
178 ==== benchmark
179 ----------
180 benchmark([caption [, label_width [, fmtstr]]]) do |x| ... end
181 benchmark([caption [, label_width [, fmtstr]]]) do array_of_Tms end
182 benchmark([caption [, label_width [, fmtstr [, labels...]]]]) do
183 ...
184 array_of_Tms
185 end
186 ----------
187
188 (({benchmark})) reports the times. In the first form the block variable x is
189 treated as a ((<Benchmark::Report>)) object, which has ((<report>)) method.
190 In the second form, each member of array_of_Tms is reported in the
191 specified form if the member is a ((<Benchmark::Tms>)) object. The
192 last form provides combined above two forms (See ((<EXAMPLE 3>))).
193
194 The following lists the meaning of each option.
195
196 :caption
197 A string ((|caption|)) is printed once before execution of the given block.
198
199 :label_width
200 An integer ((|label_width|)) is used as an offset in each report.
201
202 :fmtstr
203 An string ((|fmtstr|)) is used to format each measurement.
204 See ((<format>))
205
206 :labels
207 The rest parameters labels is used as prefix of the format to the
208 value of block, that is array_of_Tms.
209
210 ==== bm
211 ----------
212 bm([label_width [, labels ...]) do ... end
213 ----------
214
215 (({bm})) is a simpler interface of ((<benchmark>)).
216 (({bm})) acts as same as follows:
217
218 benchmark(" "*label_width + CAPTION, label_width, FMTSTR, *labels) do
219 ...
220 end
221
222 ==== bmbm
223 ----------
224 bmbm([label_width]) do |x|
225 x.item("label1") { .... }
226 ....
227 end
228 ----------
229
230 (({bmbm})) is yet another ((<benchmark>)). This utility function is
231 provited to prevent a kind of job order dependency, which is caused
232 by memory allocation and object creation. The usage is similar to
233 ((<bm>)) but has less options and does extra three things:
234
235 (1) ((*Rehearsal*)): runs all items in the job ((<list>)) to allocate
236 enough memory.
237 (2) ((*GC*)): before each ((<measure>))ment, invokes (({GC.start})) to
238 prevent the influence of previous job.
239 (3) If given ((|label_width|)) is less than the maximal width of labels
240 given as ((|item|))'s argument, the latter is used.
241 Because (({bmbm})) is a 2-pass procedure, this is possible.
242
243 (({bmbm})) returns an array which consists of Tms correspoding to each
244 (({item})).
245 ==== measure
246 ----------
247 measure([label]) do ... end
248 ----------
249
250 measure returns the times used and passed to execute the given block as a
251 Benchmark::Tms object.
252
253 ==== realtime
254 ----------
255 realtime do ... end
256 ----------
257
258 realtime returns the times passed to execute the given block.
259
260 == Benchmark::Report
261
262 === CLASS METHOD
263
264 ==== Benchmark::Report::new(width)
265 ----------
266 Benchmark::Report::new([width [, fmtstr]])
267 ----------
268
269 Usually, one doesn't have to use this method directly,
270 (({Benchmark::Report::new})) is called by ((<benchmark>)) or ((<bm>)).
271 ((|width|)) and ((|fmtstr|)) are the offset of ((|label|)) and
272 format string responsively; Both of them are used in ((<format>)).
273
274 === METHOD
275
276 ==== report
277
278 ----------
279 report(fmt, *args)
280 ----------
281
282 This method reports label and time formated by ((|fmt|)). See
283 ((<format>)) of Benchmark::Tms for formatting rule.
284
285 == Benchmark::Tms
286
287 === CLASS METHOD
288
289 == Benchmark::Job
290
291 === CLASS METHOD
292
293 ==== Benchmark::Job::new
294 ----------
295 Benchmark::Job::new(width)
296 ----------
297
298 Usually, one doesn't have to use this method directly,
299 (({Benchmark::Job::new})) is called by ((<bmbm>)).
300 ((|width|)) is a initial value for the offset ((|label|)) for formatting.
301 ((<bmbm>)) passes its argument ((|width|)) to this constructor.
302
303 === METHOD
304
305 ==== item
306 ----------
307 item(((|lable|))){ .... }
308 ----------
309
310 (({item})) registers a pair of (((|label|))) and given block as job ((<list>)).
311 ==== width
312
313 Maximum length of labels in ((<list>)) plus one.
314
315 ==== list
316
317 array of array which consists of label and jop proc.
318
319 ==== report
320
321 alias to ((<item>)).
322
323 ==== Benchmark::Tms::new
324 ----------
325 Benchmark::Tms::new([u [, s [, cu [, cs [, re [, l]]]]]])
326 ----------
327
328 returns new Benchmark::Tms object which has
329 ((|u|)) as ((<utime>)),
330 ((|s|)) as ((<stime>)),
331 ((|cu|)) as ((<cutime>))
332 ((|cs|)) as ((<cstime>)),
333 ((|re|)) as ((<real>)) and
334 ((|l|)) as ((<label>)).
335
336 The default value is assumed as 0.0 for ((|u|)), ((|s|)), ((|cu|)),
337 ((|cs|)) and ((|re|)). The default of ((|l|)) is null string ((({""}))).
338
339 ==== operator +
340 ----------
341 tms1 + tms2
342 ----------
343
344 returns a new Benchmark::Tms object as memberwise summation.
345 This method and ((<(('operator /'))>)) is useful to take statistics.
346
347 ==== operator /
348 ----------
349 tms / num
350 ----------
351
352 returns a new Benchmark::Tms object as memberwise division by ((|num|)).
353 This method and ((<operator +>)) is useful to take statistics.
354
355 ==== add
356 ----------
357 add do ... end
358 ----------
359
360 returns a new Benchmark::Tms object which is the result of additional
361 execution which is given by block.
362
363 ==== add!
364 ----------
365 add! do ... end
366 ----------
367
368 do additional execution which is given by block.
369
370 ==== format
371 ----------
372 format([fmtstr [, *args]])
373 ----------
374
375 (({format})) returns formatted string of (({self})) according to a
376 ((|fmtstr|)) like (({Kernel::format})). In addition, (({format})) accepts
377 some extentions as follows:
378 :%u
379 ((<utime>)).
380 :%y
381 ((<stime>)). (Mnemonic: y of ``s((*y*))stem'')
382 :%U
383 ((<cutime>)).
384 :%Y
385 ((<cstime>)).
386 :%t
387 ((<total>)).
388 :%r
389 ((<real>)).
390 :%n
391 ((<label>)). (Mnemonic: n of ``((*n*))ame'')
392
393 If fmtstr is not given ((<FMTSTR>)) is used as default value.
394
395 ==== utime
396
397 returns user time.
398
399 ==== stime
400
401 returns system time.
402
403 ==== cutime
404
405 returns user time of children.
406
407 ==== cstime
408
409 returns system time of children.
410
411 ==== total
412
413 returns total time, that is
414 ((<utime>)) + ((<stime>)) + ((<cutime>)) + ((<cstime>)).
415
416 ==== real
417
418 returns really passed time.
419
420 ==== label
421
422 returns label.
423
424 ==== to_a
425
426 returns a new array as follows
427
428 [label, utime, stime, cutime, cstime, real]
429
430 ==== to_s
431
432 same as (({format()})). See also ((<format>)).
433
434 == HISTORY
435 <<< benchmark.rb
436
437 == AUTHOR
438
439 Gotoken (gotoken@notwork.org).
440 =end
441
442 module Benchmark
443 BENCHMARK_VERSION = "2002-04-25"
444
445 def Benchmark::times()
446 Process::times()
447 end
448
449 def benchmark(caption = "", label_width = nil, fmtstr = nil, *labels)
450 sync = STDOUT.sync
451 STDOUT.sync = true
452 label_width ||= 0
453 fmtstr ||= FMTSTR
454 raise ArgumentError, "no block" unless iterator?
455 print caption
456 results = yield(Report.new(label_width, fmtstr))
457 Array === results and results.grep(Tms).each {|t|
458 print((labels.shift || t.label || "").ljust(label_width),
459 t.format(fmtstr))
460 }
461 STDOUT.sync = sync
462 end
463
464 def bm(label_width = 0, *labels, &blk)
465 benchmark(" "*label_width + CAPTION, label_width, FMTSTR, *labels, &blk)
466 end
467
468 def bmbm(width = 0, &blk)
469 job = Job.new(width)
470 yield(job)
471 width = job.width
472 sync = STDOUT.sync
473 STDOUT.sync = true
474
475 # rehearsal
476 print "Rehearsal "
477 puts '-'*(width+CAPTION.length - "Rehearsal ".length)
478 list = []
479 job.list.each{|label,item|
480 print(label.ljust(width))
481 res = Benchmark::measure(&item)
482 print res.format()
483 list.push res
484 }
485 sum = Tms.new; list.each{|i| sum += i}
486 ets = sum.format("total: %tsec")
487 printf("%s %s\n\n",
488 "-"*(width+CAPTION.length-ets.length-1), ets)
489
490 # take
491 print ' '*width, CAPTION
492 list = []
493 ary = []
494 job.list.each{|label,item|
495 GC::start
496 print label.ljust(width)
497 res = Benchmark::measure(&item)
498 print res.format()
499 ary.push res
500 list.push [label, res]
501 }
502
503 STDOUT.sync = sync
504 ary
505 end
506
507 def measure(label = "")
508 t0, r0 = Benchmark.times, Time.now
509 yield
510 t1, r1 = Benchmark.times, Time.now
511 Benchmark::Tms.new(t1.utime - t0.utime,
512 t1.stime - t0.stime,
513 t1.cutime - t0.cutime,
514 t1.cstime - t0.cstime,
515 r1.to_f - r0.to_f,
516 label)
517 end
518
519 def realtime(&blk)
520 Benchmark::measure(&blk).real
521 end
522
523 class Job
524 def initialize(width)
525 @width = width
526 @list = []
527 end
528
529 def item(label = "", &blk)
530 raise ArgmentError, "no block" unless block_given?
531 label.concat ' '
532 w = label.length
533 @width = w if @width < w
534 @list.push [label, blk]
535 self
536 end
537
538 alias report item
539 attr_reader :list, :width
540 end
541
542 module_function :benchmark, :measure, :realtime, :bm, :bmbm
543
544 class Report
545 def initialize(width = 0, fmtstr = nil)
546 @width, @fmtstr = width, fmtstr
547 end
548
549 def item(label = "", *fmt, &blk)
550 print label.ljust(@width)
551 res = Benchmark::measure(&blk)
552 print res.format(@fmtstr, *fmt)
553 res
554 end
555
556 alias report item
557 end
558
559 class Tms
560 CAPTION = " user system total real\n"
561 FMTSTR = "%10.6u %10.6y %10.6t %10.6r\n"
562
563 attr_reader :utime, :stime, :cutime, :cstime, :real, :total, :label
564
565 def initialize(u = 0.0, s = 0.0, cu = 0.0, cs = 0.0, real = 0.0, l = nil)
566 @utime, @stime, @cutime, @cstime, @real, @label = u, s, cu, cs, real, l
567 @total = @utime + @stime + @cutime + @cstime
568 end
569
570 def add(&blk)
571 self + Benchmark::measure(&blk)
572 end
573
574 def add!
575 t = Benchmark::measure(&blk)
576 @utime = utime + t.utime
577 @stime = stime + t.stime
578 @cutime = cutime + t.cutime
579 @cstime = cstime + t.cstime
580 @real = real + t.real
581 self
582 end
583
584 def +(x); memberwise(:+, x) end
585 def -(x); memberwise(:-, x) end
586 def *(x); memberwise(:*, x) end
587 def /(x); memberwise(:/, x) end
588
589 def format(arg0 = nil, *args)
590 fmtstr = (arg0 || FMTSTR).dup
591 fmtstr.gsub!(/(%[-+\.\d]*)n/){"#{$1}s" % label}
592 fmtstr.gsub!(/(%[-+\.\d]*)u/){"#{$1}f" % utime}
593 fmtstr.gsub!(/(%[-+\.\d]*)y/){"#{$1}f" % stime}
594 fmtstr.gsub!(/(%[-+\.\d]*)U/){"#{$1}f" % cutime}
595 fmtstr.gsub!(/(%[-+\.\d]*)Y/){"#{$1}f" % cstime}
596 fmtstr.gsub!(/(%[-+\.\d]*)t/){"#{$1}f" % total}
597 fmtstr.gsub!(/(%[-+\.\d]*)r/){"(#{$1}f)" % real}
598 arg0 ? Kernel::format(fmtstr, *args) : fmtstr
599 end
600
601 def to_s
602 format
603 end
604
605 def to_a
606 [@label, @utime, @stime, @cutime, @cstime, @real]
607 end
608
609 protected
610 def memberwise(op, x)
611 case x
612 when Benchmark::Tms
613 Benchmark::Tms.new(utime.__send__(op, x.utime),
614 stime.__send__(op, x.stime),
615 cutime.__send__(op, x.cutime),
616 cstime.__send__(op, x.cstime),
617 real.__send__(op, x.real)
618 )
619 else
620 Benchmark::Tms.new(utime.__send__(op, x),
621 stime.__send__(op, x),
622 cutime.__send__(op, x),
623 cstime.__send__(op, x),
624 real.__send__(op, x)
625 )
626 end
627 end
628 end
629
630 CAPTION = Benchmark::Tms::CAPTION
631 FMTSTR = Benchmark::Tms::FMTSTR
632 end
633
634 if __FILE__ == $0
635 include Benchmark
636
637 n = ARGV[0].to_i.nonzero? || 50000
638 puts %Q([#{n} times iterations of `a = "1"'])
639 benchmark(" " + CAPTION, 7, FMTSTR) do |x|
640 x.report("for:") {for i in 1..n; a = "1"; end} # Benchmark::measure
641 x.report("times:") {n.times do ; a = "1"; end}
642 x.report("upto:") {1.upto(n) do ; a = "1"; end}
643 end
644
645 benchmark do
646 [
647 measure{for i in 1..n; a = "1"; end}, # Benchmark::measure
648 measure{n.times do ; a = "1"; end},
649 measure{1.upto(n) do ; a = "1"; end}
650 ]
651 end
652 end