1 =begin
2
3 = net/http.rb
4
5 Copyright (c) 1999-2002 Yukihiro Matsumoto
6
7 written & maintained by Minero Aoki <aamine@loveruby.net>
8 This file is derived from "http-access.rb".
9
10 This program is free software. You can re-distribute and/or
11 modify this program under the same terms as Ruby itself,
12 Ruby Distribute License or GNU General Public License.
13
14 NOTE: You can find Japanese version of this document in
15 the doc/net directory of the standard ruby interpreter package.
16
17 $Id: http.rb,v 1.72 2002/07/11 21:33:38 aamine Exp $
18
19 == What is this module?
20
21 This module provide your program the functions to access WWW
22 documents via HTTP, Hyper Text Transfer Protocol version 1.1.
23 For details of HTTP, refer [RFC2616]
24 ((<URL:http://www.ietf.org/rfc/rfc2616.txt>)).
25
26 == Examples
27
28 === Getting Document From Server
29
30 require 'net/http'
31 Net::HTTP.start( 'some.www.server', 80 ) {|http|
32 response = http.get('/index.html')
33 puts response.body
34 }
35
36 (shorter version)
37
38 require 'net/http'
39 Net::HTTP.get_print 'some.www.server', '/index.html'
40 # or
41 Net::HTTP.get_print URI.parse('http://www.example.com/index.html')
42
43 === Posting Form Data
44
45 require 'net/http'
46 Net::HTTP.start( 'some.www.server', 80 ) {|http|
47 response = http.post('/cgi-bin/any.rhtml',
48 'querytype=subject&target=ruby')
49 }
50
51 === Accessing via Proxy
52
53 Net::HTTP.Proxy() creates http proxy class. It has same
54 methods of Net::HTTP but its instances always connect to
55 proxy, instead of given host.
56
57 require 'net/http'
58
59 $proxy_addr = 'your.proxy.addr'
60 $proxy_port = 8080
61 :
62 Net::HTTP::Proxy($proxy_addr, $proxy_port).start('some.www.server') {|http|
63 # always connect to your.proxy.addr:8080
64 :
65 }
66
67 Since Net::HTTP.Proxy() returns Net::HTTP itself when $proxy_addr is nil,
68 there's no need to change code if there's proxy or not.
69
70 === Following Redirection
71
72 require 'net/http'
73
74 def read_uri( uri_str )
75 response = Net::HTTP.get_response(URI.parse(uri_str))
76 case response
77 when Net::HTTPSuccess then response
78 when Net::HTTPRedirection then read_uri(response['location'])
79 else
80 response.error!
81 end
82 end
83
84 print read_uri('http://www.ruby-lang.org')
85
86 Net::HTTPSuccess and Net::HTTPRedirection is a HTTPResponse class.
87 All HTTPResponse objects belong to its own response class which
88 indicate HTTP result status. For details of response classes,
89 see section "HTTP Response Classes".
90
91 === Basic Authentication
92
93 require 'net/http'
94
95 req = Net::HTTP::Get.new('/need-auth.cgi')
96 req.basic_auth 'account', 'password'
97 Net::HTTP.start( 'auth.some.domain' ) {|http|
98 response = http.request(req)
99 print response.body
100 }
101
102 === HTTP Response Classes
103
104 Followings are sub classes of Net::HTTPResponse. All classes are
105 defined under the Net module. Indentation indicates inheritance.
106
107 xxx HTTPResponse
108
109 1xx HTTPInformation
110 100 HTTPContinue
111 101 HTTPSwitchProtocol
112
113 2xx HTTPSuccess
114 200 HTTPOK
115 201 HTTPCreated
116 202 HTTPAccepted
117 203 HTTPNonAuthoritativeInformation
118 204 HTTPNoContent
119 205 HTTPResetContent
120 206 HTTPPartialContent
121
122 3xx HTTPRedirection
123 300 HTTPMultipleChoice
124 301 HTTPMovedPermanently
125 302 HTTPFound
126 303 HTTPSeeOther
127 304 HTTPNotModified
128 305 HTTPUseProxy
129 307 HTTPTemporaryRedirect
130
131 4xx HTTPClientError
132 400 HTTPBadRequest
133 401 HTTPUnauthorized
134 402 HTTPPaymentRequired
135 403 HTTPForbidden
136 404 HTTPNotFound
137 405 HTTPMethodNotAllowed
138 406 HTTPNotAcceptable
139 407 HTTPProxyAuthenticationRequired
140 408 HTTPRequestTimeOut
141 409 HTTPConflict
142 410 HTTPGone
143 411 HTTPLengthRequired
144 412 HTTPPreconditionFailed
145 413 HTTPRequestEntityTooLarge
146 414 HTTPRequestURITooLong
147 415 HTTPUnsupportedMediaType
148 416 HTTPRequestedRangeNotSatisfiable
149 417 HTTPExpectationFailed
150
151 5xx HTTPServerError
152 500 HTTPInternalServerError
153 501 HTTPNotImplemented
154 502 HTTPBadGateway
155 503 HTTPServiceUnavailable
156 504 HTTPGatewayTimeOut
157 505 HTTPVersionNotSupported
158
159 xxx HTTPUnknownResponse
160
161 == Switching Net::HTTP versions
162
163 You can use net/http.rb 1.1 features (bundled with Ruby 1.6)
164 by calling HTTP.version_1_1. Calling Net::HTTP.version_1_2
165 allows you to use 1.2 features again.
166
167 # example
168 Net::HTTP.start {|http1| ...(http1 has 1.2 features)... }
169
170 Net::HTTP.version_1_1
171 Net::HTTP.start {|http2| ...(http2 has 1.1 features)... }
172
173 Net::HTTP.version_1_2
174 Net::HTTP.start {|http3| ...(http3 has 1.2 features)... }
175
176 This function is not thread-safe.
177
178 == class Net::HTTP
179
180 === Class Methods
181
182 : new( address, port = 80, proxy_addr = nil, proxy_port = nil )
183 creates a new Net::HTTP object.
184 If proxy_addr is given, creates an Net::HTTP object with proxy support.
185
186 : start( address, port = 80, proxy_addr = nil, proxy_port = nil )
187 creates a new Net::HTTP object and returns it
188 with opening HTTP session.
189
190 : start( address, port = 80, proxy_addr = nil, proxy_port = nil ) {|http| .... }
191 creates a new Net::HTTP object and gives it to the block.
192 HTTP session is kept to open while the block is exected.
193
194 This method returns the return value of the block.
195
196 : get_print( uri )
197 : get_print( address, path, port = 80 )
198 gets entity body from the target and output it to stdout.
199
200 Net::HTTP.get_print URI.parse('http://www.example.com')
201
202 : get( uri )
203 : get( address, path, port = 80 )
204 send GET request to the target and get a response.
205 This method returns a String.
206
207 print Net::HTTP.get(URI.parse('http://www.example.com'))
208
209 : get_response( uri )
210 : get_response( address, path, port = 80 )
211 send GET request to the target and get a response.
212 This method returns a Net::HTTPResponse object.
213
214 res = Net::HTTP.get_response(URI.parse('http://www.example.com'))
215 print res.body
216
217 : Proxy( address, port = 80 )
218 creates a HTTP proxy class.
219 Arguments are address/port of proxy host.
220 You can replace HTTP class with created proxy class.
221
222 If ADDRESS is nil, this method returns self (Net::HTTP).
223
224 # example
225 proxy_class = Net::HTTP::Proxy( 'proxy.foo.org', 8080 )
226 :
227 proxy_class.start( 'www.ruby-lang.org' ) {|http|
228 # connecting proxy.foo.org:8080
229 :
230 }
231
232 : proxy_class?
233 If self is HTTP, false.
234 If self is a class which was created by HTTP::Proxy(), true.
235
236 : port
237 default HTTP port (80).
238
239 === Instance Methods
240
241 : start
242 : start {|http| .... }
243 opens HTTP session.
244
245 When this method is called with block, gives a HTTP object
246 to the block and closes the HTTP session after block call finished.
247
248 : started?
249 returns true if HTTP session is started.
250
251 : address
252 the address to connect
253
254 : port
255 the port number to connect
256
257 : open_timeout
258 : open_timeout=(n)
259 seconds to wait until connection is opened.
260 If HTTP object cannot open a conection in this seconds,
261 it raises TimeoutError exception.
262
263 : read_timeout
264 : read_timeout=(n)
265 seconds to wait until reading one block (by one read(1) call).
266 If HTTP object cannot open a conection in this seconds,
267 it raises TimeoutError exception.
268
269 : finish
270 finishes HTTP session.
271 If HTTP session had not started, raises an IOError.
272
273 : proxy?
274 true if self is a HTTP proxy class
275
276 : proxy_address
277 address of proxy host. If self does not use a proxy, nil.
278
279 : proxy_port
280 port number of proxy host. If self does not use a proxy, nil.
281
282 : get( path, header = nil )
283 : get( path, header = nil ) {|str| .... }
284 gets data from PATH on the connecting host.
285 HEADER must be a Hash like { 'Accept' => '*/*', ... }.
286
287 In version 1.1, this method returns a pair of objects,
288 a Net::HTTPResponse object and entity body string.
289 In version 1.2, this method returns a Net::HTTPResponse
290 object.
291
292 If called with block, gives entity body string to the block
293 little by little.
294
295 In version 1.1, this method might raises exception for also
296 3xx (redirect). On the case you can get a HTTPResponse object
297 by "anException.response".
298 In version 1.2, this method never raises exception.
299
300 # version 1.1 (bundled with Ruby 1.6)
301 response, body = http.get( '/index.html' )
302
303 # version 1.2 (bundled with Ruby 1.7 or later)
304 response = http.get( '/index.html' )
305
306 # compatible in both version
307 response , = http.get( '/index.html' )
308 response.body
309
310 # using block
311 File.open( 'save.txt', 'w' ) {|f|
312 http.get( '/~foo/', nil ) do |str|
313 f.write str
314 end
315 }
316
317 : head( path, header = nil )
318 gets only header from PATH on the connecting host.
319 HEADER is a Hash like { 'Accept' => '*/*', ... }.
320
321 This method returns a Net::HTTPResponse object.
322
323 In version 1.1, this method might raises exception for also
324 3xx (redirect). On the case you can get a HTTPResponse object
325 by "anException.response".
326 In version 1.2, this method never raises exception.
327
328 response = nil
329 Net::HTTP.start( 'some.www.server', 80 ) {|http|
330 response = http.head( '/index.html' )
331 }
332 p response['content-type']
333
334 : post( path, data, header = nil )
335 : post( path, data, header = nil ) {|str| .... }
336 posts DATA (must be String) to PATH. HEADER must be a Hash
337 like { 'Accept' => '*/*', ... }.
338
339 In version 1.1, this method returns a pair of objects, a
340 Net::HTTPResponse object and an entity body string.
341 In version 1.2, this method returns a Net::HTTPReponse object.
342
343 If called with block, gives a part of entity body string.
344
345 In version 1.1, this method might raises exception for also
346 3xx (redirect). On the case you can get a HTTPResponse object
347 by "anException.response".
348 In version 1.2, this method never raises exception.
349
350 # version 1.1
351 response, body = http.post( '/cgi-bin/search.rb', 'query=subject&target=ruby' )
352
353 # version 1.2
354 response = http.post( '/cgi-bin/search.rb', 'query=subject&target=ruby' )
355
356 # compatible in both version
357 response , = http.post( '/cgi-bin/search.rb', 'query=subject&target=ruby' )
358
359 # using block
360 File.open( 'save.html', 'w' ) {|f|
361 http.post( '/cgi-bin/search.rb',
362 'query=subject&target=ruby' ) do |str|
363 f.write str
364 end
365 }
366
367 : request_get( path, header = nil )
368 : request_get( path, header = nil ) {|response| .... }
369 gets entity from PATH. This method returns a HTTPResponse object.
370
371 When called with block, keep connection while block is executed
372 and gives a HTTPResponse object to the block.
373
374 This method never raises Net::* exceptions.
375
376 # example
377 response = http.request_get( '/index.html' )
378 p response['content-type']
379 puts response.body # body is already read
380
381 # using block
382 http.request_get( '/index.html' ) {|response|
383 p response['content-type']
384 response.read_body do |str| # read body now
385 print str
386 end
387 }
388
389 : request_post( path, data, header = nil )
390 : request_post( path, data, header = nil ) {|response| .... }
391 posts data to PATH. This method returns a HTTPResponse object.
392
393 When called with block, gives a HTTPResponse object to the block
394 before reading entity body, with keeping connection.
395
396 This method never raises Net::* exceptions.
397
398 # example
399 response = http.post2( '/cgi-bin/nice.rb', 'datadatadata...' )
400 p response.status
401 puts response.body # body is already read
402
403 # using block
404 http.post2( '/cgi-bin/nice.rb', 'datadatadata...' ) {|response|
405 p response.status
406 p response['content-type']
407 response.read_body do |str| # read body now
408 print str
409 end
410 }
411
412 : request( request [, data] )
413 : request( request [, data] ) {|response| .... }
414 sends a HTTPRequest object REQUEST to the HTTP server.
415 This method also writes DATA string if REQUEST is a post/put request.
416 Giving DATA for get/head request causes ArgumentError.
417
418 If called with block, this method passes a HTTPResponse object to
419 the block, without reading entity body.
420
421 This method never raises Net::* exceptions.
422
423 == class Net::HTTPRequest
424
425 HTTP request class. This class wraps request header and entity path.
426 You MUST use its subclass, Net::HTTP::Get, Post, Head.
427
428 === Class Methods
429
430 : new
431 creats HTTP request object.
432
433 === Instance Methods
434
435 : self[ key ]
436 returns the header field corresponding to the case-insensitive key.
437 For example, a key of "Content-Type" might return "text/html"
438
439 : self[ key ] = val
440 sets the header field corresponding to the case-insensitive key.
441
442 : each {|name, val| .... }
443 iterates for each field name and value pair.
444
445 : basic_auth( account, password )
446 set Authorization: header for basic auth.
447
448 : range
449 returns a Range object which represents Range: header field.
450
451 : range = r
452 : set_range( i, len )
453 set Range: header from Range (arg r) or beginning index and
454 length from it (arg i&len).
455
456 : content_length
457 returns a Integer object which represents Content-Length: header field.
458
459 : content_range
460 returns a Range object which represents Content-Range: header field.
461
462 == class Net::HTTPResponse
463
464 HTTP response class. This class wraps response header and entity.
465 All arguments named KEY is case-insensitive.
466
467 === Instance Methods
468
469 : self[ key ]
470 returns the header field corresponding to the case-insensitive key.
471 For example, a key of "Content-Type" might return "text/html".
472 A key of "Content-Length" might do "2045".
473
474 More than one fields which has same names are joined with ','.
475
476 : self[ key ] = val
477 sets the header field corresponding to the case-insensitive key.
478
479 : key?( key )
480 true if key exists.
481 KEY is case insensitive.
482
483 : each {|name,value| .... }
484 iterates for each field name and value pair.
485
486 : canonical_each {|name,value| .... }
487 iterates for each "canonical" field name and value pair.
488
489 : code
490 HTTP result code string. For example, '302'.
491
492 : message
493 HTTP result message. For example, 'Not Found'.
494
495 : read_body( dest = '' )
496 gets entity body and write it into DEST using "<<" method.
497 If this method is called twice or more, nothing will be done
498 and returns first DEST.
499
500 : read_body {|str| .... }
501 gets entity body little by little and pass it to block.
502
503 : body
504 response body. If #read_body has been called, this method returns
505 arg of #read_body DEST. Else gets body as String and returns it.
506
507 =end
508
509 require 'net/protocol'
510 require 'uri'
511
512
513 module Net
514
515 class HTTPBadResponse < StandardError; end
516 class HTTPHeaderSyntaxError < StandardError; end
517
518
519 class HTTP < Protocol
520
521 HTTPVersion = '1.1'
522
523
524 #
525 # for backward compatibility
526 #
527
528 @@newimpl = true
529
530 def HTTP.version_1_2
531 @@newimpl = true
532 end
533
534 def HTTP.version_1_1
535 @@newimpl = false
536 end
537
538 def HTTP.is_version_1_2?
539 @@newimpl
540 end
541
542 def HTTP.setimplversion( obj )
543 f = @@newimpl
544 obj.instance_eval { @newimpl = f }
545 end
546 private_class_method :setimplversion
547
548
549 #
550 # short cut methods
551 #
552
553 def HTTP.get_print( arg1, arg2 = nil, port = nil )
554 if arg2
555 addr, path = arg1, arg2
556 else
557 uri = arg1
558 addr = uri.host
559 path = uri.request_uri
560 port = uri.port
561 end
562 new(addr, port || HTTP.default_port).start {|http|
563 http.get path, nil, $stdout
564 }
565 nil
566 end
567
568 def HTTP.get( arg1, arg2 = nil, arg3 = nil )
569 get_response(arg1,arg2,arg3).body
570 end
571
572 def HTTP.get_response( arg1, arg2 = nil, arg3 = nil )
573 if arg2 then
574 get_by_path(arg1, arg2, arg3)
575 else
576 get_by_uri(arg1)
577 end
578 end
579
580 def HTTP.get_by_path( addr, path, port = nil )
581 new( addr, port || HTTP.default_port ).start {|http|
582 return http.request(Get.new(path))
583 }
584 end
585 private_class_method :get_by_path
586
587 def HTTP.get_by_uri( uri )
588 # Should we allow this?
589 # uri = URI.parse(uri) unless uri.respond_to?(:host)
590 new(uri.host, uri.port).start {|http|
591 return http.request(Get.new(uri.request_uri))
592 }
593 end
594 private_class_method :get_by_uri
595
596
597 #
598 # connection
599 #
600
601 protocol_param :default_port, '80'
602 protocol_param :socket_type, '::Net::InternetMessageIO'
603
604 class << HTTP
605 def start( address, port = nil, p_addr = nil, p_port = nil, &block )
606 new( address, port, p_addr, p_port ).start( &block )
607 end
608
609 alias newobj new
610
611 def new( address, port = nil, p_addr = nil, p_port = nil )
612 obj = Proxy(p_addr, p_port).newobj(address, port)
613 setimplversion obj
614 obj
615 end
616 end
617
618 def initialize( addr, port = nil )
619 super
620 @curr_http_version = HTTPVersion
621 @seems_1_0_server = false
622 @close_on_empty_response = false
623 end
624
625 attr_accessor :close_on_empty_response
626
627 private
628
629 def do_start
630 conn_socket
631 end
632
633 def do_finish
634 disconn_socket
635 end
636
637
638 #
639 # proxy
640 #
641
642 public
643
644 # no proxy
645 @is_proxy_class = false
646 @proxy_addr = nil
647 @proxy_port = nil
648
649 def HTTP.Proxy( p_addr, p_port = nil )
650 p_addr or return self
651
652 p_port ||= port()
653 delta = ProxyDelta
654 proxyclass = Class.new(self)
655 proxyclass.module_eval {
656 include delta
657 # with proxy
658 @is_proxy_class = true
659 @proxy_address = p_addr
660 @proxy_port = p_port
661 }
662 proxyclass
663 end
664
665 class << HTTP
666 def proxy_class?
667 @is_proxy_class
668 end
669
670 attr_reader :proxy_address
671 attr_reader :proxy_port
672 end
673
674 def proxy?
675 self.class.proxy_class?
676 end
677
678 def proxy_address
679 self.class.proxy_address
680 end
681
682 def proxy_port
683 self.class.proxy_port
684 end
685
686 alias proxyaddr proxy_address
687 alias proxyport proxy_port
688
689 private
690
691 # no proxy
692
693 def conn_address
694 address
695 end
696
697 def conn_port
698 port
699 end
700
701 def edit_path( path )
702 path
703 end
704
705 module ProxyDelta
706 private
707
708 # with proxy
709
710 def conn_address
711 proxy_address
712 end
713
714 def conn_port
715 proxy_port
716 end
717
718 def edit_path( path )
719 'http://' + addr_port() + path
720 end
721 end
722
723
724 #
725 # http operations
726 #
727
728 public
729
730 def get( path, initheader = nil, dest = nil, &block )
731 res = nil
732 request( Get.new(path,initheader) ) {|res|
733 res.read_body dest, &block
734 }
735 unless @newimpl then
736 res.value
737 return res, res.body
738 end
739
740 res
741 end
742
743 def head( path, initheader = nil )
744 res = request( Head.new(path,initheader) )
745 @newimpl or res.value
746 res
747 end
748
749 def post( path, data, initheader = nil, dest = nil, &block )
750 res = nil
751 request( Post.new(path,initheader), data ) {|res|
752 res.read_body dest, &block
753 }
754 unless @newimpl then
755 res.value
756 return res, res.body
757 end
758
759 res
760 end
761
762 def put( path, data, initheader = nil )
763 res = request( Put.new(path,initheader), data )
764 @newimpl or res.value
765 res
766 end
767
768
769 def request_get( path, initheader = nil, &block )
770 request Get.new(path,initheader), &block
771 end
772
773 def request_head( path, initheader = nil, &block )
774 request Head.new(path,initheader), &block
775 end
776
777 def request_post( path, data, initheader = nil, &block )
778 request Post.new(path,initheader), data, &block
779 end
780
781 def request_put( path, data, initheader = nil, &block )
782 request Put.new(path,initheader), data, &block
783 end
784
785 alias get2 request_get
786 alias head2 request_head
787 alias post2 request_post
788 alias put2 request_put
789
790
791 def send_request( name, path, body = nil, header = nil )
792 r = HTTPGenericRequest.new( name, (body ? true : false), true,
793 path, header )
794 request r, body
795 end
796
797
798 def request( req, body = nil, &block )
799 unless started? then
800 start {
801 req['connection'] = 'close'
802 return request(req, body, &block)
803 }
804 end
805
806 begin_transport req
807 req.exec @socket, @curr_http_version, edit_path(req.path), body
808 begin
809 res = HTTPResponse.read_new(@socket)
810 end while HTTPContinue === res
811 res.reading_body(@socket, req.response_body_permitted?) {
812 yield res if block_given?
813 }
814 end_transport req, res
815
816 res
817 end
818
819 private
820
821 def begin_transport( req )
822 if @socket.closed? then
823 reconn_socket
824 end
825 if @seems_1_0_server then
826 req['connection'] = 'close'
827 end
828 if not req.response_body_permitted? and @close_on_empty_response then
829 req['connection'] = 'close'
830 end
831 req['host'] = addr_port()
832 end
833
834 def end_transport( req, res )
835 @curr_http_version = res.http_version
836
837 if not res.body and @close_on_empty_response then
838 D 'Conn close'
839 @socket.close
840 elsif keep_alive? req, res then
841 D 'Conn keep-alive'
842 if @socket.closed? then
843 D 'Conn (but seems 1.0 server)'
844 @seems_1_0_server = true
845 end
846 else
847 D 'Conn close'
848 @socket.close
849 end
850 end
851
852 def keep_alive?( req, res )
853 /close/i === req['connection'].to_s and return false
854 @seems_1_0_server and return false
855
856 /keep-alive/i === res['connection'].to_s and return true
857 /close/i === res['connection'].to_s and return false
858 /keep-alive/i === res['proxy-connection'].to_s and return true
859 /close/i === res['proxy-connection'].to_s and return false
860
861 @curr_http_version == '1.1' and return true
862 false
863 end
864
865
866 #
867 # utils
868 #
869
870 private
871
872 def addr_port
873 address + (port == HTTP.default_port ? '' : ":#{port}")
874 end
875
876 def D( msg )
877 if @debug_output then
878 @debug_output << msg
879 @debug_output << "\n"
880 end
881 end
882
883 end
884
885 HTTPSession = HTTP
886
887
888 ###
889 ### header
890 ###
891
892 module HTTPHeader
893
894 def size
895 @header.size
896 end
897
898 alias length size
899
900 def []( key )
901 @header[ key.downcase ]
902 end
903
904 def []=( key, val )
905 @header[ key.downcase ] = val
906 end
907
908 def each_header( &block )
909 @header.each( &block )
910 end
911
912 alias each each_header
913
914 def each_key( &block )
915 @header.each_key( &block )
916 end
917
918 def each_value( &block )
919 @header.each_value( &block )
920 end
921
922 def delete( key )
923 @header.delete key.downcase
924 end
925
926 def key?( key )
927 @header.key? key.downcase
928 end
929
930 def to_hash
931 @header.dup
932 end
933
934 def canonical_each
935 @header.each do |k,v|
936 yield canonical(k), v
937 end
938 end
939
940 def canonical( k )
941 k.split('-').collect {|i| i.capitalize }.join('-')
942 end
943
944 def range
945 s = @header['range'] or return nil
946 s.split(',').collect {|spec|
947 m = /bytes\s*=\s*(\d+)?\s*-\s*(\d+)?/i.match(spec) or
948 raise HTTPHeaderSyntaxError, "wrong Range: #{spec}"
949 d1 = m[1].to_i
950 d2 = m[2].to_i
951 if m[1] and m[2] then d1..d2
952 elsif m[1] then d1..-1
953 elsif m[2] then -d2..-1
954 else
955 raise HTTPHeaderSyntaxError, 'range is not specified'
956 end
957 }
958 end
959
960 def range=( r, fin = nil )
961 r = (r ... r + fin) if fin
962
963 case r
964 when Numeric
965 s = r > 0 ? "0-#{r - 1}" : "-#{-r}"
966 when Range
967 first = r.first
968 last = r.last
969 if r.exclude_end? then
970 last -= 1
971 end
972
973 if last == -1 then
974 s = first > 0 ? "#{first}-" : "-#{-first}"
975 else
976 first >= 0 or raise HTTPHeaderSyntaxError, 'range.first is negative'
977 last > 0 or raise HTTPHeaderSyntaxError, 'range.last is negative'
978 first < last or raise HTTPHeaderSyntaxError, 'must be .first < .last'
979 s = "#{first}-#{last}"
980 end
981 else
982 raise TypeError, 'Range/Integer is required'
983 end
984
985 @header['range'] = "bytes=#{s}"
986 r
987 end
988
989 alias set_range range=
990
991 def content_length
992 s = @header['content-length'] or return nil
993 m = /\d+/.match(s) or
994 raise HTTPHeaderSyntaxError, 'wrong Content-Length format'
995 m[0].to_i
996 end
997
998 def chunked?
999 s = @header['transfer-encoding']
1000 (s and /(?:\A|[^\-\w])chunked(?:[^\-\w]|\z)/i === s) ? true : false
1001 end
1002
1003 def content_range
1004 s = @header['content-range'] or return nil
1005 m = %r<bytes\s+(\d+)-(\d+)/(?:\d+|\*)>i.match(s) or
1006 raise HTTPHeaderSyntaxError, 'wrong Content-Range format'
1007 m[1].to_i .. m[2].to_i + 1
1008 end
1009
1010 def range_length
1011 r = self.content_range
1012 r and r.length
1013 end
1014
1015 def basic_auth( acc, pass )
1016 @header['authorization'] = 'Basic ' + ["#{acc}:#{pass}"].pack('m').strip
1017 end
1018
1019 end
1020
1021
1022 ###
1023 ### request
1024 ###
1025
1026 class HTTPGenericRequest
1027
1028 include HTTPHeader
1029
1030 def initialize( m, reqbody, resbody, path, initheader = nil )
1031 @method = m
1032 @request_has_body = reqbody
1033 @response_has_body = resbody
1034 @path = path
1035
1036 @header = tmp = {}
1037 return unless initheader
1038 initheader.each do |k,v|
1039 key = k.downcase
1040 if tmp.key? key then
1041 $stderr.puts "WARNING: duplicated HTTP header: #{k}" if $VERBOSE
1042 end
1043 tmp[ key ] = v.strip
1044 end
1045 tmp['accept'] ||= '*/*'
1046 end
1047
1048 attr_reader :method
1049 attr_reader :path
1050
1051 def inspect
1052 "\#<#{self.class} #{@method}>"
1053 end
1054
1055 def request_body_permitted?
1056 @request_has_body
1057 end
1058
1059 def response_body_permitted?
1060 @response_has_body
1061 end
1062
1063 alias body_exist? response_body_permitted?
1064
1065 #
1066 # write
1067 #
1068
1069 # internal use only
1070 def exec( sock, ver, path, body )
1071 if body then
1072 check_body_premitted
1073 send_request_with_body sock, ver, path, body
1074 else
1075 request sock, ver, path
1076 end
1077 end
1078
1079 private
1080
1081 def check_body_premitted
1082 request_body_permitted? or
1083 raise ArgumentError, 'HTTP request body is not premitted'
1084 end
1085
1086 def send_request_with_body( sock, ver, path, body )
1087 if block_given? then
1088 ac = Accumulator.new
1089 yield ac # must be yield, DO NOT USE block.call
1090 data = ac.terminate
1091 else
1092 data = body
1093 end
1094 @header['content-length'] = data.size.to_s
1095 @header.delete 'transfer-encoding'
1096
1097 unless @header['content-type'] then
1098 $stderr.puts 'Content-Type did not set; using application/x-www-form-urlencoded' if $VERBOSE
1099 @header['content-type'] = 'application/x-www-form-urlencoded'
1100 end
1101
1102 request sock, ver, path
1103 sock.write data
1104 end
1105
1106 def request( sock, ver, path )
1107 sock.writeline sprintf('%s %s HTTP/%s', @method, path, ver)
1108 canonical_each do |k,v|
1109 sock.writeline k + ': ' + v
1110 end
1111 sock.writeline ''
1112 end
1113
1114 end
1115
1116
1117 class HTTPRequest < HTTPGenericRequest
1118
1119 def initialize( path, initheader = nil )
1120 super self.class::METHOD,
1121 self.class::REQUEST_HAS_BODY,
1122 self.class::RESPONSE_HAS_BODY,
1123 path, initheader
1124 end
1125
1126 end
1127
1128
1129 class HTTP
1130
1131 class Get < HTTPRequest
1132 METHOD = 'GET'
1133 REQUEST_HAS_BODY = false
1134 RESPONSE_HAS_BODY = true
1135 end
1136
1137 class Head < HTTPRequest
1138 METHOD = 'HEAD'
1139 REQUEST_HAS_BODY = false
1140 RESPONSE_HAS_BODY = false
1141 end
1142
1143 class Post < HTTPRequest
1144 METHOD = 'POST'
1145 REQUEST_HAS_BODY = true
1146 RESPONSE_HAS_BODY = true
1147 end
1148
1149 class Put < HTTPRequest
1150 METHOD = 'PUT'
1151 REQUEST_HAS_BODY = true
1152 RESPONSE_HAS_BODY = true
1153 end
1154
1155 end
1156
1157
1158
1159 ###
1160 ### response
1161 ###
1162
1163 class HTTPResponse
1164 # predefine HTTPResponse class to allow inheritance
1165
1166 def self.body_permitted?
1167 self::HAS_BODY
1168 end
1169
1170 def self.exception_type
1171 self::EXCEPTION_TYPE
1172 end
1173 end
1174
1175
1176 class HTTPUnknownResponse < HTTPResponse
1177 HAS_BODY = true
1178 EXCEPTION_TYPE = ProtocolError
1179 end
1180 class HTTPInformation < HTTPResponse # 1xx
1181 HAS_BODY = false
1182 EXCEPTION_TYPE = ProtocolError
1183 end
1184 class HTTPSuccess < HTTPResponse # 2xx
1185 HAS_BODY = true
1186 EXCEPTION_TYPE = ProtocolError
1187 end
1188 class HTTPRedirection < HTTPResponse # 3xx
1189 HAS_BODY = true
1190 EXCEPTION_TYPE = ProtoRetriableError
1191 end
1192 class HTTPClientError < HTTPResponse # 4xx
1193 HAS_BODY = true
1194 EXCEPTION_TYPE = ProtoFatalError
1195 end
1196 class HTTPServerError < HTTPResponse # 5xx
1197 HAS_BODY = true
1198 EXCEPTION_TYPE = ProtoServerError
1199 end
1200
1201 class HTTPContinue < HTTPInformation # 100
1202 HAS_BODY = false
1203 end
1204 class HTTPSwitchProtocol < HTTPInformation # 101
1205 HAS_BODY = false
1206 end
1207
1208 class HTTPOK < HTTPSuccess # 200
1209 HAS_BODY = true
1210 end
1211 class HTTPCreated < HTTPSuccess # 201
1212 HAS_BODY = true
1213 end
1214 class HTTPAccepted < HTTPSuccess # 202
1215 HAS_BODY = true
1216 end
1217 class HTTPNonAuthoritativeInformation < HTTPSuccess # 203
1218 HAS_BODY = true
1219 end
1220 class HTTPNoContent < HTTPSuccess # 204
1221 HAS_BODY = false
1222 end
1223 class HTTPResetContent < HTTPSuccess # 205
1224 HAS_BODY = false
1225 end
1226 class HTTPPartialContent < HTTPSuccess # 206
1227 HAS_BODY = true
1228 end
1229
1230 class HTTPMultipleChoice < HTTPRedirection # 300
1231 HAS_BODY = true
1232 end
1233 class HTTPMovedPermanently < HTTPRedirection # 301
1234 HAS_BODY = true
1235 end
1236 class HTTPFound < HTTPRedirection # 302
1237 HAS_BODY = true
1238 end
1239 HTTPMovedTemporarily = HTTPFound
1240 class HTTPSeeOther < HTTPRedirection # 303
1241 HAS_BODY = true
1242 end
1243 class HTTPNotModified < HTTPRedirection # 304
1244 HAS_BODY = false
1245 end
1246 class HTTPUseProxy < HTTPRedirection # 305
1247 HAS_BODY = false
1248 end
1249 # 306 unused
1250 class HTTPTemporaryRedirect < HTTPRedirection # 307
1251 HAS_BODY = true
1252 end
1253
1254 class HTTPBadRequest < HTTPClientError # 400
1255 HAS_BODY = true
1256 end
1257 class HTTPUnauthorized < HTTPClientError # 401
1258 HAS_BODY = true
1259 end
1260 class HTTPPaymentRequired < HTTPClientError # 402
1261 HAS_BODY = true
1262 end
1263 class HTTPForbidden < HTTPClientError # 403
1264 HAS_BODY = true
1265 end
1266 class HTTPNotFound < HTTPClientError # 404
1267 HAS_BODY = true
1268 end
1269 class HTTPMethodNotAllowed < HTTPClientError # 405
1270 HAS_BODY = true
1271 end
1272 class HTTPNotAcceptable < HTTPClientError # 406
1273 HAS_BODY = true
1274 end
1275 class HTTPProxyAuthenticationRequired < HTTPClientError # 407
1276 HAS_BODY = true
1277 end
1278 class HTTPRequestTimeOut < HTTPClientError # 408
1279 HAS_BODY = true
1280 end
1281 class HTTPConflict < HTTPClientError # 409
1282 HAS_BODY = true
1283 end
1284 class HTTPGone < HTTPClientError # 410
1285 HAS_BODY = true
1286 end
1287 class HTTPLengthRequired < HTTPClientError # 411
1288 HAS_BODY = true
1289 end
1290 class HTTPPreconditionFailed < HTTPClientError # 412
1291 HAS_BODY = true
1292 end
1293 class HTTPRequestEntityTooLarge < HTTPClientError # 413
1294 HAS_BODY = true
1295 end
1296 class HTTPRequestURITooLong < HTTPClientError # 414
1297 HAS_BODY = true
1298 end
1299 HTTPRequestURITooLarge = HTTPRequestURITooLong
1300 class HTTPUnsupportedMediaType < HTTPClientError # 415
1301 HAS_BODY = true
1302 end
1303 class HTTPRequestedRangeNotSatisfiable < HTTPClientError # 416
1304 HAS_BODY = true
1305 end
1306 class HTTPExpectationFailed < HTTPClientError # 417
1307 HAS_BODY = true
1308 end
1309
1310 class HTTPInternalServerError < HTTPServerError # 500
1311 HAS_BODY = true
1312 end
1313 class HTTPNotImplemented < HTTPServerError # 501
1314 HAS_BODY = true
1315 end
1316 class HTTPBadGateway < HTTPServerError # 502
1317 HAS_BODY = true
1318 end
1319 class HTTPServiceUnavailable < HTTPServerError # 503
1320 HAS_BODY = true
1321 end
1322 class HTTPGatewayTimeOut < HTTPServerError # 504
1323 HAS_BODY = true
1324 end
1325 class HTTPVersionNotSupported < HTTPServerError # 505
1326 HAS_BODY = true
1327 end
1328
1329
1330 class HTTPResponse # redefine
1331
1332 CODE_CLASS_TO_OBJ = {
1333 '1' => HTTPInformation,
1334 '2' => HTTPSuccess,
1335 '3' => HTTPRedirection,
1336 '4' => HTTPClientError,
1337 '5' => HTTPServerError
1338 }
1339 CODE_TO_OBJ = {
1340 '100' => HTTPContinue,
1341 '101' => HTTPSwitchProtocol,
1342
1343 '200' => HTTPOK,
1344 '201' => HTTPCreated,
1345 '202' => HTTPAccepted,
1346 '203' => HTTPNonAuthoritativeInformation,
1347 '204' => HTTPNoContent,
1348 '205' => HTTPResetContent,
1349 '206' => HTTPPartialContent,
1350
1351 '300' => HTTPMultipleChoice,
1352 '301' => HTTPMovedPermanently,
1353 '302' => HTTPFound,
1354 '303' => HTTPSeeOther,
1355 '304' => HTTPNotModified,
1356 '305' => HTTPUseProxy,
1357 '307' => HTTPTemporaryRedirect,
1358
1359 '400' => HTTPBadRequest,
1360 '401' => HTTPUnauthorized,
1361 '402' => HTTPPaymentRequired,
1362 '403' => HTTPForbidden,
1363 '404' => HTTPNotFound,
1364 '405' => HTTPMethodNotAllowed,
1365 '406' => HTTPNotAcceptable,
1366 '407' => HTTPProxyAuthenticationRequired,
1367 '408' => HTTPRequestTimeOut,
1368 '409' => HTTPConflict,
1369 '410' => HTTPGone,
1370 '411' => HTTPLengthRequired,
1371 '412' => HTTPPreconditionFailed,
1372 '413' => HTTPRequestEntityTooLarge,
1373 '414' => HTTPRequestURITooLong,
1374 '415' => HTTPUnsupportedMediaType,
1375 '416' => HTTPRequestedRangeNotSatisfiable,
1376 '417' => HTTPExpectationFailed,
1377
1378 '501' => HTTPInternalServerError,
1379 '501' => HTTPNotImplemented,
1380 '502' => HTTPBadGateway,
1381 '503' => HTTPServiceUnavailable,
1382 '504' => HTTPGatewayTimeOut,
1383 '505' => HTTPVersionNotSupported
1384 }
1385
1386
1387 class << self
1388
1389 def read_new( sock )
1390 httpv, code, msg = read_status_line(sock)
1391 res = response_class(code).new(httpv, code, msg)
1392 each_response_header(sock) do |k,v|
1393 if res.key? k then
1394 res[k] << ', ' << v
1395 else
1396 res[k] = v
1397 end
1398 end
1399
1400 res
1401 end
1402
1403 private
1404
1405 def read_status_line( sock )
1406 str = sock.readline
1407 m = /\AHTTP(?:\/(\d+\.\d+))?\s+(\d\d\d)\s*(.*)\z/in.match(str) or
1408 raise HTTPBadResponse, "wrong status line: #{str.dump}"
1409 m.to_a[1,3]
1410 end
1411
1412 def response_class( code )
1413 CODE_TO_OBJ[code] or
1414 CODE_CLASS_TO_OBJ[code[0,1]] or
1415 HTTPUnknownResponse
1416 end
1417
1418 def each_response_header( sock )
1419 while true do
1420 line = sock.readuntil( "\n", true ) # ignore EOF
1421 line.sub!( /\s+\z/, '' ) # don't use chop!
1422 break if line.empty?
1423
1424 m = /\A([^:]+):\s*/.match(line) or
1425 raise HTTPBadResponse, 'wrong header line format'
1426 yield m[1], m.post_match
1427 end
1428 end
1429
1430 end
1431
1432
1433 include HTTPHeader
1434
1435 def initialize( httpv, code, msg )
1436 @http_version = httpv
1437 @code = code
1438 @message = msg
1439
1440 @header = {}
1441 @body = nil
1442 @read = false
1443 end
1444
1445 attr_reader :http_version
1446 attr_reader :code
1447 attr_reader :message
1448 alias msg message
1449
1450 def inspect
1451 "#<#{self.class} #{@code} readbody=#{@read}>"
1452 end
1453
1454 #
1455 # response <-> exception relationship
1456 #
1457
1458 def code_type
1459 self.class
1460 end
1461
1462 def error!
1463 raise error_type().new(@code + ' ' + @message.dump, self)
1464 end
1465
1466 def error_type
1467 self.class::EXCEPTION_TYPE
1468 end
1469
1470 def value
1471 HTTPSuccess === self or error!
1472 end
1473
1474 #
1475 # header (for backward compatibility only; DO NOT USE)
1476 #
1477
1478 def response
1479 self
1480 end
1481
1482 alias header response
1483 alias read_header response
1484
1485 #
1486 # body
1487 #
1488
1489 # internal use only
1490 def reading_body( sock, reqmethodallowbody )
1491 @socket = sock
1492 @body_exist = reqmethodallowbody && self.class.body_permitted?
1493 yield
1494 self.body
1495 @socket = nil
1496 end
1497
1498 def read_body( dest = nil, &block )
1499 if @read then
1500 (dest or block) and
1501 raise IOError, "#{self.class}\#read_body called twice"
1502 return @body
1503 end
1504
1505 to = procdest(dest, block)
1506 stream_check
1507 if @body_exist then
1508 read_body_0 to
1509 @body = to
1510 else
1511 @body = nil
1512 end
1513 @read = true
1514
1515 @body
1516 end
1517
1518 alias body read_body
1519 alias entity read_body
1520
1521 private
1522
1523 def read_body_0( dest )
1524 if chunked? then
1525 read_chunked dest
1526 else
1527 clen = content_length
1528 if clen then
1529 @socket.read clen, dest, true # ignore EOF
1530 else
1531 clen = range_length
1532 if clen then
1533 @socket.read clen, dest
1534 else
1535 @socket.read_all dest
1536 end
1537 end
1538 end
1539 end
1540
1541 def read_chunked( dest )
1542 len = nil
1543 total = 0
1544
1545 while true do
1546 line = @socket.readline
1547 m = /[0-9a-fA-F]+/.match(line)
1548 m or raise HTTPBadResponse, "wrong chunk size line: #{line}"
1549 len = m[0].hex
1550 break if len == 0
1551 @socket.read len, dest; total += len
1552 @socket.read 2 # \r\n
1553 end
1554 until @socket.readline.empty? do
1555 # none
1556 end
1557 end
1558
1559 def stream_check
1560 @socket.closed? and raise IOError, 'try to read body out of block'
1561 end
1562
1563 def procdest( dest, block )
1564 (dest and block) and
1565 raise ArgumentError, 'both of arg and block are given for HTTP method'
1566 if block then
1567 ReadAdapter.new(block)
1568 else
1569 dest || ''
1570 end
1571 end
1572
1573 end
1574
1575
1576
1577 # for backward compatibility
1578
1579 module NetPrivate
1580 HTTPResponse = ::Net::HTTPResponse
1581 HTTPGenericRequest = ::Net::HTTPGenericRequest
1582 HTTPRequest = ::Net::HTTPRequest
1583 HTTPHeader = ::Net::HTTPHeader
1584 end
1585 HTTPInformationCode = HTTPInformation
1586 HTTPSuccessCode = HTTPSuccess
1587 HTTPRedirectionCode = HTTPRedirection
1588 HTTPRetriableCode = HTTPRedirection
1589 HTTPClientErrorCode = HTTPClientError
1590 HTTPFatalErrorCode = HTTPClientError
1591 HTTPServerErrorCode = HTTPServerError
1592 HTTPResponceReceiver = HTTPResponse
1593
1594 end # module Net