IntenseWebs/freeipa
3
0
Fork 0
mirror of https://salsa.debian.org/freeipa-team/freeipa.git synced 2024-12-28 18:01:23 -06:00
Code Issues Packages Projects Releases Wiki Activity
d2e3af88eb
freeipa/ipatests/test_ipapython/test_cookie.py

479 lines
19 KiB
Python
Raw Normal View History

Compliant client side session cookie behavior In summary this patch does: * Follow the defined rules for cookies when: - receiving a cookie (process the attributes) - storing a cookie (store cookie + attributes) - sending a cookie + validate the cookie domain against the request URL + validate the cookie path against the request URL + validate the cookie expiration + if valid then send only the cookie, no attribtues * Modifies how a request URL is stored during a XMLRPC request/response sequence. * Refactors a bit of the request/response logic to allow for making the decision whether to send a session cookie instead of full Kerberous auth easier. * The server now includes expiration information in the session cookie it sends to the client. The server always had the information available to prevent using an expired session cookie. Now that expiration timestamp is returned to the client as well and now the client will not send an expired session cookie back to the server. * Adds a new module and unit test for cookies (see below) Formerly we were always returning the session cookie no matter what the domain or path was in the URL. We were also sending the cookie attributes which are for the client only (used to determine if to return a cookie). The attributes are not meant to be sent to the server and the previous behavior was a protocol violation. We also were not checking the cookie expiration. Cookie library issues: We need a library to create, parse, manipulate and format cookies both in a client context and a server context. Core Python has two cookie libraries, Cookie.py and cookielib.py. Why did we add a new cookie module instead of using either of these two core Python libaries? Cookie.py is designed for server side generation but can be used to parse cookies on the client. It's the library we were using in the server. However when I tried to use it in the client I discovered it has some serious bugs. There are 7 defined cookie elements, it fails to correctly parse 3 of the 7 elements which makes it unusable because we depend on those elements. Since Cookie.py was designed for server side cookie processing it's not hard to understand how fails to correctly parse a cookie because that's a client side need. (Cookie.py also has an awkward baroque API and is missing some useful functionality we would have to build on top of it). cookielib.py is designed for client side. It's fully featured and obeys all the RFC's. It would be great to use however it's tightly coupled with another core library, urllib2.py. The http request and response objects must be urllib2 objects. But we don't use urllib2, rather we use httplib because xmlrpclib uses httplib. I don't see a reason why a cookie library should be so tightly coupled to a protocol library, but it is and that means we can't use it (I tried to just pick some isolated entrypoints for our use but I kept hitting interaction/dependency problems). I decided to solve the cookie library problems by writing a minimal cookie library that does what we need and no more than that. It is a new module in ipapython shared by both client and server and comes with a new unit test. The module has plenty of documentation, no need to repeat it here. Request URL issues: We also had problems in rpc.py whereby information from the request which is needed when we process the response is not available. Most important was the requesting URL. It turns out that the way the class and object relationships are structured it's impossible to get this information. Someone else must have run into the same issue because there was a routine called reconstruct_url() which attempted to recreate the request URL from other available information. Unfortunately reconstruct_url() was not callable from inside the response handler. So I decided to store the information in the thread context and when the request is received extract it from the thread context. It's perhaps not an ideal solution but we do similar things elsewhere so at least it's consistent. I removed the reconstruct_url() function because the exact information is now in the context and trying to apply heuristics to recreate the url is probably not robust. Ticket https://fedorahosted.org/freeipa/ticket/3022
2012-12-04 17:20:17 -06:00
# Authors:
# John Dennis <jdennis@redhat.com>
#
# Copyright (C) 2012 Red Hat
# see file 'COPYING' for use and warranty information
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
import unittest
import time
import datetime
Cookie Expires date should be locale insensitive The Expires attribute in a cookie is supposed to follow the RFC 822 (superseded by RFC 1123) date format. That format includes a weekday abbreviation (e.g. Tue) which must be in English according to the RFC's. ipapython/cookie.py has methods to parse and format the Expires attribute but they were based on strptime() and strftime() which respects the locale. If a non-English locale is in effect the wrong date string will be produced and/or it won't be able to parse the date string. The fix is to use the date parsing and formatting functions from email.utils which specifically follow the RFC's and are not locale sensitive. This patch also updates the unit test to use email.utils as well. The patch should be applied to the following branches: Ticket: https://fedorahosted.org/freeipa/ticket/3313
2012-12-19 11:47:46 -06:00
import email.utils
Compliant client side session cookie behavior In summary this patch does: * Follow the defined rules for cookies when: - receiving a cookie (process the attributes) - storing a cookie (store cookie + attributes) - sending a cookie + validate the cookie domain against the request URL + validate the cookie path against the request URL + validate the cookie expiration + if valid then send only the cookie, no attribtues * Modifies how a request URL is stored during a XMLRPC request/response sequence. * Refactors a bit of the request/response logic to allow for making the decision whether to send a session cookie instead of full Kerberous auth easier. * The server now includes expiration information in the session cookie it sends to the client. The server always had the information available to prevent using an expired session cookie. Now that expiration timestamp is returned to the client as well and now the client will not send an expired session cookie back to the server. * Adds a new module and unit test for cookies (see below) Formerly we were always returning the session cookie no matter what the domain or path was in the URL. We were also sending the cookie attributes which are for the client only (used to determine if to return a cookie). The attributes are not meant to be sent to the server and the previous behavior was a protocol violation. We also were not checking the cookie expiration. Cookie library issues: We need a library to create, parse, manipulate and format cookies both in a client context and a server context. Core Python has two cookie libraries, Cookie.py and cookielib.py. Why did we add a new cookie module instead of using either of these two core Python libaries? Cookie.py is designed for server side generation but can be used to parse cookies on the client. It's the library we were using in the server. However when I tried to use it in the client I discovered it has some serious bugs. There are 7 defined cookie elements, it fails to correctly parse 3 of the 7 elements which makes it unusable because we depend on those elements. Since Cookie.py was designed for server side cookie processing it's not hard to understand how fails to correctly parse a cookie because that's a client side need. (Cookie.py also has an awkward baroque API and is missing some useful functionality we would have to build on top of it). cookielib.py is designed for client side. It's fully featured and obeys all the RFC's. It would be great to use however it's tightly coupled with another core library, urllib2.py. The http request and response objects must be urllib2 objects. But we don't use urllib2, rather we use httplib because xmlrpclib uses httplib. I don't see a reason why a cookie library should be so tightly coupled to a protocol library, but it is and that means we can't use it (I tried to just pick some isolated entrypoints for our use but I kept hitting interaction/dependency problems). I decided to solve the cookie library problems by writing a minimal cookie library that does what we need and no more than that. It is a new module in ipapython shared by both client and server and comes with a new unit test. The module has plenty of documentation, no need to repeat it here. Request URL issues: We also had problems in rpc.py whereby information from the request which is needed when we process the response is not available. Most important was the requesting URL. It turns out that the way the class and object relationships are structured it's impossible to get this information. Someone else must have run into the same issue because there was a routine called reconstruct_url() which attempted to recreate the request URL from other available information. Unfortunately reconstruct_url() was not callable from inside the response handler. So I decided to store the information in the thread context and when the request is received extract it from the thread context. It's perhaps not an ideal solution but we do similar things elsewhere so at least it's consistent. I removed the reconstruct_url() function because the exact information is now in the context and trying to apply heuristics to recreate the url is probably not robust. Ticket https://fedorahosted.org/freeipa/ticket/3022
2012-12-04 17:20:17 -06:00
import calendar
from ipapython.cookie import Cookie
class TestParse(unittest.TestCase):
def test_parse(self):
# Empty string
s = ''
cookies = Cookie.parse(s)
self.assertEqual(len(cookies), 0)
# Invalid single token
s = 'color'
with self.assertRaises(ValueError):
cookies = Cookie.parse(s)
# Invalid single token that's keyword
s = 'HttpOnly'
with self.assertRaises(ValueError):
cookies = Cookie.parse(s)
# Invalid key/value pair whose key is a keyword
s = 'domain=example.com'
with self.assertRaises(ValueError):
cookies = Cookie.parse(s)
# 1 cookie with name/value
s = 'color=blue'
cookies = Cookie.parse(s)
self.assertEqual(len(cookies), 1)
cookie = cookies[0]
self.assertEqual(cookie.key, 'color')
self.assertEqual(cookie.value, 'blue')
self.assertEqual(cookie.domain, None)
self.assertEqual(cookie.path, None)
self.assertEqual(cookie.max_age, None)
self.assertEqual(cookie.expires, None)
self.assertEqual(cookie.secure, None)
self.assertEqual(cookie.httponly, None)
self.assertEqual(str(cookie), "color=blue")
self.assertEqual(cookie.http_cookie(), "color=blue;")
# 1 cookie with whose value is quoted
# Use "get by name" utility to extract specific cookie
s = 'color="blue"'
cookie = Cookie.get_named_cookie_from_string(s, 'color')
self.assertIsNotNone(cookie)
self.assertIsNotNone(cookie, Cookie)
self.assertEqual(cookie.key, 'color')
self.assertEqual(cookie.value, 'blue')
self.assertEqual(cookie.domain, None)
self.assertEqual(cookie.path, None)
self.assertEqual(cookie.max_age, None)
self.assertEqual(cookie.expires, None)
self.assertEqual(cookie.secure, None)
self.assertEqual(cookie.httponly, None)
self.assertEqual(str(cookie), "color=blue")
self.assertEqual(cookie.http_cookie(), "color=blue;")
# 1 cookie with name/value and domain, path attributes.
# Change up the whitespace a bit.
s = 'color =blue; domain= example.com ; path = /toplevel '
cookies = Cookie.parse(s)
self.assertEqual(len(cookies), 1)
cookie = cookies[0]
self.assertEqual(cookie.key, 'color')
self.assertEqual(cookie.value, 'blue')
self.assertEqual(cookie.domain, 'example.com')
self.assertEqual(cookie.path, '/toplevel')
self.assertEqual(cookie.max_age, None)
self.assertEqual(cookie.expires, None)
self.assertEqual(cookie.secure, None)
self.assertEqual(cookie.httponly, None)
self.assertEqual(str(cookie), "color=blue; Domain=example.com; Path=/toplevel")
self.assertEqual(cookie.http_cookie(), "color=blue;")
# 2 cookies, various attributes
s = 'color=blue; Max-Age=3600; temperature=hot; HttpOnly'
cookies = Cookie.parse(s)
self.assertEqual(len(cookies), 2)
cookie = cookies[0]
self.assertEqual(cookie.key, 'color')
self.assertEqual(cookie.value, 'blue')
self.assertEqual(cookie.domain, None)
self.assertEqual(cookie.path, None)
self.assertEqual(cookie.max_age, 3600)
self.assertEqual(cookie.expires, None)
self.assertEqual(cookie.secure, None)
self.assertEqual(cookie.httponly, None)
self.assertEqual(str(cookie), "color=blue; Max-Age=3600")
self.assertEqual(cookie.http_cookie(), "color=blue;")
cookie = cookies[1]
self.assertEqual(cookie.key, 'temperature')
self.assertEqual(cookie.value, 'hot')
self.assertEqual(cookie.domain, None)
self.assertEqual(cookie.path, None)
self.assertEqual(cookie.max_age, None)
self.assertEqual(cookie.expires, None)
self.assertEqual(cookie.secure, None)
self.assertEqual(cookie.httponly, True)
self.assertEqual(str(cookie), "temperature=hot; HttpOnly")
self.assertEqual(cookie.http_cookie(), "temperature=hot;")
class TestExpires(unittest.TestCase):
def setUp(self):
# Force microseconds to zero because cookie timestamps only have second resolution
self.now = datetime.datetime.utcnow().replace(microsecond=0)
self.now_timestamp = calendar.timegm(self.now.utctimetuple())
Cookie Expires date should be locale insensitive The Expires attribute in a cookie is supposed to follow the RFC 822 (superseded by RFC 1123) date format. That format includes a weekday abbreviation (e.g. Tue) which must be in English according to the RFC's. ipapython/cookie.py has methods to parse and format the Expires attribute but they were based on strptime() and strftime() which respects the locale. If a non-English locale is in effect the wrong date string will be produced and/or it won't be able to parse the date string. The fix is to use the date parsing and formatting functions from email.utils which specifically follow the RFC's and are not locale sensitive. This patch also updates the unit test to use email.utils as well. The patch should be applied to the following branches: Ticket: https://fedorahosted.org/freeipa/ticket/3313
2012-12-19 11:47:46 -06:00
self.now_string = email.utils.formatdate(self.now_timestamp, usegmt=True)
Compliant client side session cookie behavior In summary this patch does: * Follow the defined rules for cookies when: - receiving a cookie (process the attributes) - storing a cookie (store cookie + attributes) - sending a cookie + validate the cookie domain against the request URL + validate the cookie path against the request URL + validate the cookie expiration + if valid then send only the cookie, no attribtues * Modifies how a request URL is stored during a XMLRPC request/response sequence. * Refactors a bit of the request/response logic to allow for making the decision whether to send a session cookie instead of full Kerberous auth easier. * The server now includes expiration information in the session cookie it sends to the client. The server always had the information available to prevent using an expired session cookie. Now that expiration timestamp is returned to the client as well and now the client will not send an expired session cookie back to the server. * Adds a new module and unit test for cookies (see below) Formerly we were always returning the session cookie no matter what the domain or path was in the URL. We were also sending the cookie attributes which are for the client only (used to determine if to return a cookie). The attributes are not meant to be sent to the server and the previous behavior was a protocol violation. We also were not checking the cookie expiration. Cookie library issues: We need a library to create, parse, manipulate and format cookies both in a client context and a server context. Core Python has two cookie libraries, Cookie.py and cookielib.py. Why did we add a new cookie module instead of using either of these two core Python libaries? Cookie.py is designed for server side generation but can be used to parse cookies on the client. It's the library we were using in the server. However when I tried to use it in the client I discovered it has some serious bugs. There are 7 defined cookie elements, it fails to correctly parse 3 of the 7 elements which makes it unusable because we depend on those elements. Since Cookie.py was designed for server side cookie processing it's not hard to understand how fails to correctly parse a cookie because that's a client side need. (Cookie.py also has an awkward baroque API and is missing some useful functionality we would have to build on top of it). cookielib.py is designed for client side. It's fully featured and obeys all the RFC's. It would be great to use however it's tightly coupled with another core library, urllib2.py. The http request and response objects must be urllib2 objects. But we don't use urllib2, rather we use httplib because xmlrpclib uses httplib. I don't see a reason why a cookie library should be so tightly coupled to a protocol library, but it is and that means we can't use it (I tried to just pick some isolated entrypoints for our use but I kept hitting interaction/dependency problems). I decided to solve the cookie library problems by writing a minimal cookie library that does what we need and no more than that. It is a new module in ipapython shared by both client and server and comes with a new unit test. The module has plenty of documentation, no need to repeat it here. Request URL issues: We also had problems in rpc.py whereby information from the request which is needed when we process the response is not available. Most important was the requesting URL. It turns out that the way the class and object relationships are structured it's impossible to get this information. Someone else must have run into the same issue because there was a routine called reconstruct_url() which attempted to recreate the request URL from other available information. Unfortunately reconstruct_url() was not callable from inside the response handler. So I decided to store the information in the thread context and when the request is received extract it from the thread context. It's perhaps not an ideal solution but we do similar things elsewhere so at least it's consistent. I removed the reconstruct_url() function because the exact information is now in the context and trying to apply heuristics to recreate the url is probably not robust. Ticket https://fedorahosted.org/freeipa/ticket/3022
2012-12-04 17:20:17 -06:00
self.max_age = 3600 # 1 hour
self.age_expiration = self.now + datetime.timedelta(seconds=self.max_age)
Cookie Expires date should be locale insensitive The Expires attribute in a cookie is supposed to follow the RFC 822 (superseded by RFC 1123) date format. That format includes a weekday abbreviation (e.g. Tue) which must be in English according to the RFC's. ipapython/cookie.py has methods to parse and format the Expires attribute but they were based on strptime() and strftime() which respects the locale. If a non-English locale is in effect the wrong date string will be produced and/or it won't be able to parse the date string. The fix is to use the date parsing and formatting functions from email.utils which specifically follow the RFC's and are not locale sensitive. This patch also updates the unit test to use email.utils as well. The patch should be applied to the following branches: Ticket: https://fedorahosted.org/freeipa/ticket/3313
2012-12-19 11:47:46 -06:00
self.age_timestamp = calendar.timegm(self.age_expiration.utctimetuple())
self.age_string = email.utils.formatdate(self.age_timestamp, usegmt=True)
Compliant client side session cookie behavior In summary this patch does: * Follow the defined rules for cookies when: - receiving a cookie (process the attributes) - storing a cookie (store cookie + attributes) - sending a cookie + validate the cookie domain against the request URL + validate the cookie path against the request URL + validate the cookie expiration + if valid then send only the cookie, no attribtues * Modifies how a request URL is stored during a XMLRPC request/response sequence. * Refactors a bit of the request/response logic to allow for making the decision whether to send a session cookie instead of full Kerberous auth easier. * The server now includes expiration information in the session cookie it sends to the client. The server always had the information available to prevent using an expired session cookie. Now that expiration timestamp is returned to the client as well and now the client will not send an expired session cookie back to the server. * Adds a new module and unit test for cookies (see below) Formerly we were always returning the session cookie no matter what the domain or path was in the URL. We were also sending the cookie attributes which are for the client only (used to determine if to return a cookie). The attributes are not meant to be sent to the server and the previous behavior was a protocol violation. We also were not checking the cookie expiration. Cookie library issues: We need a library to create, parse, manipulate and format cookies both in a client context and a server context. Core Python has two cookie libraries, Cookie.py and cookielib.py. Why did we add a new cookie module instead of using either of these two core Python libaries? Cookie.py is designed for server side generation but can be used to parse cookies on the client. It's the library we were using in the server. However when I tried to use it in the client I discovered it has some serious bugs. There are 7 defined cookie elements, it fails to correctly parse 3 of the 7 elements which makes it unusable because we depend on those elements. Since Cookie.py was designed for server side cookie processing it's not hard to understand how fails to correctly parse a cookie because that's a client side need. (Cookie.py also has an awkward baroque API and is missing some useful functionality we would have to build on top of it). cookielib.py is designed for client side. It's fully featured and obeys all the RFC's. It would be great to use however it's tightly coupled with another core library, urllib2.py. The http request and response objects must be urllib2 objects. But we don't use urllib2, rather we use httplib because xmlrpclib uses httplib. I don't see a reason why a cookie library should be so tightly coupled to a protocol library, but it is and that means we can't use it (I tried to just pick some isolated entrypoints for our use but I kept hitting interaction/dependency problems). I decided to solve the cookie library problems by writing a minimal cookie library that does what we need and no more than that. It is a new module in ipapython shared by both client and server and comes with a new unit test. The module has plenty of documentation, no need to repeat it here. Request URL issues: We also had problems in rpc.py whereby information from the request which is needed when we process the response is not available. Most important was the requesting URL. It turns out that the way the class and object relationships are structured it's impossible to get this information. Someone else must have run into the same issue because there was a routine called reconstruct_url() which attempted to recreate the request URL from other available information. Unfortunately reconstruct_url() was not callable from inside the response handler. So I decided to store the information in the thread context and when the request is received extract it from the thread context. It's perhaps not an ideal solution but we do similar things elsewhere so at least it's consistent. I removed the reconstruct_url() function because the exact information is now in the context and trying to apply heuristics to recreate the url is probably not robust. Ticket https://fedorahosted.org/freeipa/ticket/3022
2012-12-04 17:20:17 -06:00
self.expires = self.now + datetime.timedelta(days=1) # 1 day
self.expires_timestamp = calendar.timegm(self.expires.utctimetuple())
Cookie Expires date should be locale insensitive The Expires attribute in a cookie is supposed to follow the RFC 822 (superseded by RFC 1123) date format. That format includes a weekday abbreviation (e.g. Tue) which must be in English according to the RFC's. ipapython/cookie.py has methods to parse and format the Expires attribute but they were based on strptime() and strftime() which respects the locale. If a non-English locale is in effect the wrong date string will be produced and/or it won't be able to parse the date string. The fix is to use the date parsing and formatting functions from email.utils which specifically follow the RFC's and are not locale sensitive. This patch also updates the unit test to use email.utils as well. The patch should be applied to the following branches: Ticket: https://fedorahosted.org/freeipa/ticket/3313
2012-12-19 11:47:46 -06:00
self.expires_string = email.utils.formatdate(self.expires_timestamp, usegmt=True)
Compliant client side session cookie behavior In summary this patch does: * Follow the defined rules for cookies when: - receiving a cookie (process the attributes) - storing a cookie (store cookie + attributes) - sending a cookie + validate the cookie domain against the request URL + validate the cookie path against the request URL + validate the cookie expiration + if valid then send only the cookie, no attribtues * Modifies how a request URL is stored during a XMLRPC request/response sequence. * Refactors a bit of the request/response logic to allow for making the decision whether to send a session cookie instead of full Kerberous auth easier. * The server now includes expiration information in the session cookie it sends to the client. The server always had the information available to prevent using an expired session cookie. Now that expiration timestamp is returned to the client as well and now the client will not send an expired session cookie back to the server. * Adds a new module and unit test for cookies (see below) Formerly we were always returning the session cookie no matter what the domain or path was in the URL. We were also sending the cookie attributes which are for the client only (used to determine if to return a cookie). The attributes are not meant to be sent to the server and the previous behavior was a protocol violation. We also were not checking the cookie expiration. Cookie library issues: We need a library to create, parse, manipulate and format cookies both in a client context and a server context. Core Python has two cookie libraries, Cookie.py and cookielib.py. Why did we add a new cookie module instead of using either of these two core Python libaries? Cookie.py is designed for server side generation but can be used to parse cookies on the client. It's the library we were using in the server. However when I tried to use it in the client I discovered it has some serious bugs. There are 7 defined cookie elements, it fails to correctly parse 3 of the 7 elements which makes it unusable because we depend on those elements. Since Cookie.py was designed for server side cookie processing it's not hard to understand how fails to correctly parse a cookie because that's a client side need. (Cookie.py also has an awkward baroque API and is missing some useful functionality we would have to build on top of it). cookielib.py is designed for client side. It's fully featured and obeys all the RFC's. It would be great to use however it's tightly coupled with another core library, urllib2.py. The http request and response objects must be urllib2 objects. But we don't use urllib2, rather we use httplib because xmlrpclib uses httplib. I don't see a reason why a cookie library should be so tightly coupled to a protocol library, but it is and that means we can't use it (I tried to just pick some isolated entrypoints for our use but I kept hitting interaction/dependency problems). I decided to solve the cookie library problems by writing a minimal cookie library that does what we need and no more than that. It is a new module in ipapython shared by both client and server and comes with a new unit test. The module has plenty of documentation, no need to repeat it here. Request URL issues: We also had problems in rpc.py whereby information from the request which is needed when we process the response is not available. Most important was the requesting URL. It turns out that the way the class and object relationships are structured it's impossible to get this information. Someone else must have run into the same issue because there was a routine called reconstruct_url() which attempted to recreate the request URL from other available information. Unfortunately reconstruct_url() was not callable from inside the response handler. So I decided to store the information in the thread context and when the request is received extract it from the thread context. It's perhaps not an ideal solution but we do similar things elsewhere so at least it's consistent. I removed the reconstruct_url() function because the exact information is now in the context and trying to apply heuristics to recreate the url is probably not robust. Ticket https://fedorahosted.org/freeipa/ticket/3022
2012-12-04 17:20:17 -06:00
def test_expires(self):
# 1 cookie with name/value and no Max-Age and no Expires
s = 'color=blue;'
cookies = Cookie.parse(s)
self.assertEqual(len(cookies), 1)
cookie = cookies[0]
# Force timestamp to known value
cookie.timestamp = self.now
self.assertEqual(cookie.key, 'color')
self.assertEqual(cookie.value, 'blue')
self.assertEqual(cookie.domain, None)
self.assertEqual(cookie.path, None)
self.assertEqual(cookie.max_age, None)
self.assertEqual(cookie.expires, None)
self.assertEqual(cookie.secure, None)
self.assertEqual(cookie.httponly, None)
self.assertEqual(str(cookie), "color=blue")
self.assertEqual(cookie.get_expiration(), None)
# Normalize
self.assertEqual(cookie.normalize_expiration(), None)
self.assertEqual(cookie.max_age, None)
self.assertEqual(cookie.expires, None)
self.assertEqual(str(cookie), "color=blue")
# 1 cookie with name/value and Max-Age
s = 'color=blue; max-age=%d' % (self.max_age)
cookies = Cookie.parse(s)
self.assertEqual(len(cookies), 1)
cookie = cookies[0]
# Force timestamp to known value
cookie.timestamp = self.now
self.assertEqual(cookie.key, 'color')
self.assertEqual(cookie.value, 'blue')
self.assertEqual(cookie.domain, None)
self.assertEqual(cookie.path, None)
self.assertEqual(cookie.max_age, self.max_age)
self.assertEqual(cookie.expires, None)
self.assertEqual(cookie.secure, None)
self.assertEqual(cookie.httponly, None)
self.assertEqual(str(cookie), "color=blue; Max-Age=%d" % (self.max_age))
self.assertEqual(cookie.get_expiration(), self.age_expiration)
# Normalize
self.assertEqual(cookie.normalize_expiration(), self.age_expiration)
self.assertEqual(cookie.max_age, None)
self.assertEqual(cookie.expires, self.age_expiration)
self.assertEqual(str(cookie), "color=blue; Expires=%s" % (self.age_string))
# 1 cookie with name/value and Expires
s = 'color=blue; Expires=%s' % (self.expires_string)
cookies = Cookie.parse(s)
self.assertEqual(len(cookies), 1)
cookie = cookies[0]
# Force timestamp to known value
cookie.timestamp = self.now
self.assertEqual(cookie.key, 'color')
self.assertEqual(cookie.value, 'blue')
self.assertEqual(cookie.domain, None)
self.assertEqual(cookie.path, None)
self.assertEqual(cookie.max_age, None)
self.assertEqual(cookie.expires, self.expires)
self.assertEqual(cookie.secure, None)
self.assertEqual(cookie.httponly, None)
self.assertEqual(str(cookie), "color=blue; Expires=%s" % (self.expires_string))
self.assertEqual(cookie.get_expiration(), self.expires)
# Normalize
self.assertEqual(cookie.normalize_expiration(), self.expires)
self.assertEqual(cookie.max_age, None)
self.assertEqual(cookie.expires, self.expires)
self.assertEqual(str(cookie), "color=blue; Expires=%s" % (self.expires_string))
# 1 cookie with name/value witht both Max-Age and Expires, Max-Age takes precedence
s = 'color=blue; Expires=%s; max-age=%d' % (self.expires_string, self.max_age)
cookies = Cookie.parse(s)
self.assertEqual(len(cookies), 1)
cookie = cookies[0]
# Force timestamp to known value
cookie.timestamp = self.now
self.assertEqual(cookie.key, 'color')
self.assertEqual(cookie.value, 'blue')
self.assertEqual(cookie.domain, None)
self.assertEqual(cookie.path, None)
self.assertEqual(cookie.max_age, self.max_age)
self.assertEqual(cookie.expires, self.expires)
self.assertEqual(cookie.secure, None)
self.assertEqual(cookie.httponly, None)
self.assertEqual(str(cookie), "color=blue; Max-Age=%d; Expires=%s" % (self.max_age, self.expires_string))
self.assertEqual(cookie.get_expiration(), self.age_expiration)
# Normalize
self.assertEqual(cookie.normalize_expiration(), self.age_expiration)
self.assertEqual(cookie.max_age, None)
self.assertEqual(cookie.expires, self.age_expiration)
self.assertEqual(str(cookie), "color=blue; Expires=%s" % (self.age_string))
# Verify different types can be assigned to the timestamp and
# expires attribute.
cookie = Cookie('color', 'blue')
cookie.timestamp = self.now
self.assertEqual(cookie.timestamp, self.now)
cookie.timestamp = self.now_timestamp
self.assertEqual(cookie.timestamp, self.now)
cookie.timestamp = self.now_string
self.assertEqual(cookie.timestamp, self.now)
self.assertEqual(cookie.expires, None)
cookie.expires = self.expires
self.assertEqual(cookie.expires, self.expires)
cookie.expires = self.expires_timestamp
self.assertEqual(cookie.expires, self.expires)
cookie.expires = self.expires_string
self.assertEqual(cookie.expires, self.expires)
class TestInvalidAttributes(unittest.TestCase):
def test_invalid(self):
# Invalid Max-Age
s = 'color=blue; Max-Age=over-the-hill'
with self.assertRaises(ValueError):
cookies = Cookie.parse(s)
cookie = Cookie('color', 'blue')
with self.assertRaises(ValueError):
cookie.max_age = 'over-the-hill'
# Invalid Expires
s = 'color=blue; Expires=Sun, 06 Xxx 1994 08:49:37 GMT'
with self.assertRaises(ValueError):
cookies = Cookie.parse(s)
cookie = Cookie('color', 'blue')
with self.assertRaises(ValueError):
cookie.expires = 'Sun, 06 Xxx 1994 08:49:37 GMT'
class TestAttributes(unittest.TestCase):
def test_attributes(self):
cookie = Cookie('color', 'blue')
self.assertEqual(cookie.key, 'color')
self.assertEqual(cookie.value, 'blue')
self.assertEqual(cookie.domain, None)
self.assertEqual(cookie.path, None)
self.assertEqual(cookie.max_age, None)
self.assertEqual(cookie.expires, None)
self.assertEqual(cookie.secure, None)
self.assertEqual(cookie.httponly, None)
cookie.domain = 'example.com'
self.assertEqual(cookie.domain, 'example.com')
cookie.domain = None
self.assertEqual(cookie.domain, None)
cookie.path = '/toplevel'
self.assertEqual(cookie.path, '/toplevel')
cookie.path = None
self.assertEqual(cookie.path, None)
cookie.max_age = 400
self.assertEqual(cookie.max_age, 400)
cookie.max_age = None
self.assertEqual(cookie.max_age, None)
cookie.expires = 'Sun, 06 Nov 1994 08:49:37 GMT'
self.assertEqual(cookie.expires, datetime.datetime(1994, 11, 6, 8, 49, 37))
cookie.expires = None
self.assertEqual(cookie.expires, None)
cookie.secure = True
self.assertEqual(cookie.secure, True)
self.assertEqual(str(cookie), "color=blue; Secure")
cookie.secure = False
self.assertEqual(cookie.secure, False)
self.assertEqual(str(cookie), "color=blue")
cookie.secure = None
self.assertEqual(cookie.secure, None)
self.assertEqual(str(cookie), "color=blue")
cookie.httponly = True
self.assertEqual(cookie.httponly, True)
self.assertEqual(str(cookie), "color=blue; HttpOnly")
cookie.httponly = False
self.assertEqual(cookie.httponly, False)
self.assertEqual(str(cookie), "color=blue")
cookie.httponly = None
self.assertEqual(cookie.httponly, None)
self.assertEqual(str(cookie), "color=blue")
class TestHTTPReturn(unittest.TestCase):
def setUp(self):
self.url = 'http://www.foo.bar.com/one/two'
def test_no_attributes(self):
cookie = Cookie('color', 'blue')
self.assertTrue(cookie.http_return_ok(self.url))
def test_domain(self):
cookie = Cookie('color', 'blue', domain='www.foo.bar.com')
self.assertTrue(cookie.http_return_ok(self.url))
cookie = Cookie('color', 'blue', domain='.foo.bar.com')
self.assertTrue(cookie.http_return_ok(self.url))
cookie = Cookie('color', 'blue', domain='.bar.com')
self.assertTrue(cookie.http_return_ok(self.url))
cookie = Cookie('color', 'blue', domain='bar.com')
with self.assertRaises(Cookie.URLMismatch):
self.assertTrue(cookie.http_return_ok(self.url))
cookie = Cookie('color', 'blue', domain='bogus.com')
with self.assertRaises(Cookie.URLMismatch):
self.assertTrue(cookie.http_return_ok(self.url))
cookie = Cookie('color', 'blue', domain='www.foo.bar.com')
with self.assertRaises(Cookie.URLMismatch):
self.assertTrue(cookie.http_return_ok('http://192.168.1.1/one/two'))
def test_path(self):
cookie = Cookie('color', 'blue')
self.assertTrue(cookie.http_return_ok(self.url))
cookie = Cookie('color', 'blue', path='/')
self.assertTrue(cookie.http_return_ok(self.url))
cookie = Cookie('color', 'blue', path='/one')
self.assertTrue(cookie.http_return_ok(self.url))
cookie = Cookie('color', 'blue', path='/oneX')
with self.assertRaises(Cookie.URLMismatch):
self.assertTrue(cookie.http_return_ok(self.url))
def test_expires(self):
now = datetime.datetime.utcnow().replace(microsecond=0)
# expires 1 day from now
expires = now + datetime.timedelta(days=1)
cookie = Cookie('color', 'blue', expires=expires)
self.assertTrue(cookie.http_return_ok(self.url))
# expired 1 day ago
expires = now + datetime.timedelta(days=-1)
cookie = Cookie('color', 'blue', expires=expires)
with self.assertRaises(Cookie.Expired):
self.assertTrue(cookie.http_return_ok(self.url))
def test_httponly(self):
cookie = Cookie('color', 'blue', httponly=True)
self.assertTrue(cookie.http_return_ok('http://example.com'))
self.assertTrue(cookie.http_return_ok('https://example.com'))
with self.assertRaises(Cookie.URLMismatch):
self.assertTrue(cookie.http_return_ok('ftp://example.com'))
def test_secure(self):
cookie = Cookie('color', 'blue', secure=True)
self.assertTrue(cookie.http_return_ok('https://Xexample.com'))
with self.assertRaises(Cookie.URLMismatch):
self.assertTrue(cookie.http_return_ok('http://Xexample.com'))
class TestNormalization(unittest.TestCase):
def setUp(self):
# Force microseconds to zero because cookie timestamps only have second resolution
self.now = datetime.datetime.utcnow().replace(microsecond=0)
self.now_timestamp = calendar.timegm(self.now.utctimetuple())
Cookie Expires date should be locale insensitive The Expires attribute in a cookie is supposed to follow the RFC 822 (superseded by RFC 1123) date format. That format includes a weekday abbreviation (e.g. Tue) which must be in English according to the RFC's. ipapython/cookie.py has methods to parse and format the Expires attribute but they were based on strptime() and strftime() which respects the locale. If a non-English locale is in effect the wrong date string will be produced and/or it won't be able to parse the date string. The fix is to use the date parsing and formatting functions from email.utils which specifically follow the RFC's and are not locale sensitive. This patch also updates the unit test to use email.utils as well. The patch should be applied to the following branches: Ticket: https://fedorahosted.org/freeipa/ticket/3313
2012-12-19 11:47:46 -06:00
self.now_string = email.utils.formatdate(self.now_timestamp, usegmt=True)
Compliant client side session cookie behavior In summary this patch does: * Follow the defined rules for cookies when: - receiving a cookie (process the attributes) - storing a cookie (store cookie + attributes) - sending a cookie + validate the cookie domain against the request URL + validate the cookie path against the request URL + validate the cookie expiration + if valid then send only the cookie, no attribtues * Modifies how a request URL is stored during a XMLRPC request/response sequence. * Refactors a bit of the request/response logic to allow for making the decision whether to send a session cookie instead of full Kerberous auth easier. * The server now includes expiration information in the session cookie it sends to the client. The server always had the information available to prevent using an expired session cookie. Now that expiration timestamp is returned to the client as well and now the client will not send an expired session cookie back to the server. * Adds a new module and unit test for cookies (see below) Formerly we were always returning the session cookie no matter what the domain or path was in the URL. We were also sending the cookie attributes which are for the client only (used to determine if to return a cookie). The attributes are not meant to be sent to the server and the previous behavior was a protocol violation. We also were not checking the cookie expiration. Cookie library issues: We need a library to create, parse, manipulate and format cookies both in a client context and a server context. Core Python has two cookie libraries, Cookie.py and cookielib.py. Why did we add a new cookie module instead of using either of these two core Python libaries? Cookie.py is designed for server side generation but can be used to parse cookies on the client. It's the library we were using in the server. However when I tried to use it in the client I discovered it has some serious bugs. There are 7 defined cookie elements, it fails to correctly parse 3 of the 7 elements which makes it unusable because we depend on those elements. Since Cookie.py was designed for server side cookie processing it's not hard to understand how fails to correctly parse a cookie because that's a client side need. (Cookie.py also has an awkward baroque API and is missing some useful functionality we would have to build on top of it). cookielib.py is designed for client side. It's fully featured and obeys all the RFC's. It would be great to use however it's tightly coupled with another core library, urllib2.py. The http request and response objects must be urllib2 objects. But we don't use urllib2, rather we use httplib because xmlrpclib uses httplib. I don't see a reason why a cookie library should be so tightly coupled to a protocol library, but it is and that means we can't use it (I tried to just pick some isolated entrypoints for our use but I kept hitting interaction/dependency problems). I decided to solve the cookie library problems by writing a minimal cookie library that does what we need and no more than that. It is a new module in ipapython shared by both client and server and comes with a new unit test. The module has plenty of documentation, no need to repeat it here. Request URL issues: We also had problems in rpc.py whereby information from the request which is needed when we process the response is not available. Most important was the requesting URL. It turns out that the way the class and object relationships are structured it's impossible to get this information. Someone else must have run into the same issue because there was a routine called reconstruct_url() which attempted to recreate the request URL from other available information. Unfortunately reconstruct_url() was not callable from inside the response handler. So I decided to store the information in the thread context and when the request is received extract it from the thread context. It's perhaps not an ideal solution but we do similar things elsewhere so at least it's consistent. I removed the reconstruct_url() function because the exact information is now in the context and trying to apply heuristics to recreate the url is probably not robust. Ticket https://fedorahosted.org/freeipa/ticket/3022
2012-12-04 17:20:17 -06:00
self.max_age = 3600 # 1 hour
self.age_expiration = self.now + datetime.timedelta(seconds=self.max_age)
Cookie Expires date should be locale insensitive The Expires attribute in a cookie is supposed to follow the RFC 822 (superseded by RFC 1123) date format. That format includes a weekday abbreviation (e.g. Tue) which must be in English according to the RFC's. ipapython/cookie.py has methods to parse and format the Expires attribute but they were based on strptime() and strftime() which respects the locale. If a non-English locale is in effect the wrong date string will be produced and/or it won't be able to parse the date string. The fix is to use the date parsing and formatting functions from email.utils which specifically follow the RFC's and are not locale sensitive. This patch also updates the unit test to use email.utils as well. The patch should be applied to the following branches: Ticket: https://fedorahosted.org/freeipa/ticket/3313
2012-12-19 11:47:46 -06:00
self.age_timestamp = calendar.timegm(self.age_expiration.utctimetuple())
self.age_string = email.utils.formatdate(self.age_timestamp, usegmt=True)
Compliant client side session cookie behavior In summary this patch does: * Follow the defined rules for cookies when: - receiving a cookie (process the attributes) - storing a cookie (store cookie + attributes) - sending a cookie + validate the cookie domain against the request URL + validate the cookie path against the request URL + validate the cookie expiration + if valid then send only the cookie, no attribtues * Modifies how a request URL is stored during a XMLRPC request/response sequence. * Refactors a bit of the request/response logic to allow for making the decision whether to send a session cookie instead of full Kerberous auth easier. * The server now includes expiration information in the session cookie it sends to the client. The server always had the information available to prevent using an expired session cookie. Now that expiration timestamp is returned to the client as well and now the client will not send an expired session cookie back to the server. * Adds a new module and unit test for cookies (see below) Formerly we were always returning the session cookie no matter what the domain or path was in the URL. We were also sending the cookie attributes which are for the client only (used to determine if to return a cookie). The attributes are not meant to be sent to the server and the previous behavior was a protocol violation. We also were not checking the cookie expiration. Cookie library issues: We need a library to create, parse, manipulate and format cookies both in a client context and a server context. Core Python has two cookie libraries, Cookie.py and cookielib.py. Why did we add a new cookie module instead of using either of these two core Python libaries? Cookie.py is designed for server side generation but can be used to parse cookies on the client. It's the library we were using in the server. However when I tried to use it in the client I discovered it has some serious bugs. There are 7 defined cookie elements, it fails to correctly parse 3 of the 7 elements which makes it unusable because we depend on those elements. Since Cookie.py was designed for server side cookie processing it's not hard to understand how fails to correctly parse a cookie because that's a client side need. (Cookie.py also has an awkward baroque API and is missing some useful functionality we would have to build on top of it). cookielib.py is designed for client side. It's fully featured and obeys all the RFC's. It would be great to use however it's tightly coupled with another core library, urllib2.py. The http request and response objects must be urllib2 objects. But we don't use urllib2, rather we use httplib because xmlrpclib uses httplib. I don't see a reason why a cookie library should be so tightly coupled to a protocol library, but it is and that means we can't use it (I tried to just pick some isolated entrypoints for our use but I kept hitting interaction/dependency problems). I decided to solve the cookie library problems by writing a minimal cookie library that does what we need and no more than that. It is a new module in ipapython shared by both client and server and comes with a new unit test. The module has plenty of documentation, no need to repeat it here. Request URL issues: We also had problems in rpc.py whereby information from the request which is needed when we process the response is not available. Most important was the requesting URL. It turns out that the way the class and object relationships are structured it's impossible to get this information. Someone else must have run into the same issue because there was a routine called reconstruct_url() which attempted to recreate the request URL from other available information. Unfortunately reconstruct_url() was not callable from inside the response handler. So I decided to store the information in the thread context and when the request is received extract it from the thread context. It's perhaps not an ideal solution but we do similar things elsewhere so at least it's consistent. I removed the reconstruct_url() function because the exact information is now in the context and trying to apply heuristics to recreate the url is probably not robust. Ticket https://fedorahosted.org/freeipa/ticket/3022
2012-12-04 17:20:17 -06:00
self.expires = self.now + datetime.timedelta(days=1) # 1 day
self.expires_timestamp = calendar.timegm(self.expires.utctimetuple())
Cookie Expires date should be locale insensitive The Expires attribute in a cookie is supposed to follow the RFC 822 (superseded by RFC 1123) date format. That format includes a weekday abbreviation (e.g. Tue) which must be in English according to the RFC's. ipapython/cookie.py has methods to parse and format the Expires attribute but they were based on strptime() and strftime() which respects the locale. If a non-English locale is in effect the wrong date string will be produced and/or it won't be able to parse the date string. The fix is to use the date parsing and formatting functions from email.utils which specifically follow the RFC's and are not locale sensitive. This patch also updates the unit test to use email.utils as well. The patch should be applied to the following branches: Ticket: https://fedorahosted.org/freeipa/ticket/3313
2012-12-19 11:47:46 -06:00
self.expires_string = email.utils.formatdate(self.expires_timestamp, usegmt=True)
Compliant client side session cookie behavior In summary this patch does: * Follow the defined rules for cookies when: - receiving a cookie (process the attributes) - storing a cookie (store cookie + attributes) - sending a cookie + validate the cookie domain against the request URL + validate the cookie path against the request URL + validate the cookie expiration + if valid then send only the cookie, no attribtues * Modifies how a request URL is stored during a XMLRPC request/response sequence. * Refactors a bit of the request/response logic to allow for making the decision whether to send a session cookie instead of full Kerberous auth easier. * The server now includes expiration information in the session cookie it sends to the client. The server always had the information available to prevent using an expired session cookie. Now that expiration timestamp is returned to the client as well and now the client will not send an expired session cookie back to the server. * Adds a new module and unit test for cookies (see below) Formerly we were always returning the session cookie no matter what the domain or path was in the URL. We were also sending the cookie attributes which are for the client only (used to determine if to return a cookie). The attributes are not meant to be sent to the server and the previous behavior was a protocol violation. We also were not checking the cookie expiration. Cookie library issues: We need a library to create, parse, manipulate and format cookies both in a client context and a server context. Core Python has two cookie libraries, Cookie.py and cookielib.py. Why did we add a new cookie module instead of using either of these two core Python libaries? Cookie.py is designed for server side generation but can be used to parse cookies on the client. It's the library we were using in the server. However when I tried to use it in the client I discovered it has some serious bugs. There are 7 defined cookie elements, it fails to correctly parse 3 of the 7 elements which makes it unusable because we depend on those elements. Since Cookie.py was designed for server side cookie processing it's not hard to understand how fails to correctly parse a cookie because that's a client side need. (Cookie.py also has an awkward baroque API and is missing some useful functionality we would have to build on top of it). cookielib.py is designed for client side. It's fully featured and obeys all the RFC's. It would be great to use however it's tightly coupled with another core library, urllib2.py. The http request and response objects must be urllib2 objects. But we don't use urllib2, rather we use httplib because xmlrpclib uses httplib. I don't see a reason why a cookie library should be so tightly coupled to a protocol library, but it is and that means we can't use it (I tried to just pick some isolated entrypoints for our use but I kept hitting interaction/dependency problems). I decided to solve the cookie library problems by writing a minimal cookie library that does what we need and no more than that. It is a new module in ipapython shared by both client and server and comes with a new unit test. The module has plenty of documentation, no need to repeat it here. Request URL issues: We also had problems in rpc.py whereby information from the request which is needed when we process the response is not available. Most important was the requesting URL. It turns out that the way the class and object relationships are structured it's impossible to get this information. Someone else must have run into the same issue because there was a routine called reconstruct_url() which attempted to recreate the request URL from other available information. Unfortunately reconstruct_url() was not callable from inside the response handler. So I decided to store the information in the thread context and when the request is received extract it from the thread context. It's perhaps not an ideal solution but we do similar things elsewhere so at least it's consistent. I removed the reconstruct_url() function because the exact information is now in the context and trying to apply heuristics to recreate the url is probably not robust. Ticket https://fedorahosted.org/freeipa/ticket/3022
2012-12-04 17:20:17 -06:00
def test_path_normalization(self):
self.assertEqual(Cookie.normalize_url_path(''), '/')
self.assertEqual(Cookie.normalize_url_path('foo'), '/')
self.assertEqual(Cookie.normalize_url_path('foo/'), '/')
self.assertEqual(Cookie.normalize_url_path('/foo'), '/')
self.assertEqual(Cookie.normalize_url_path('/foo/'), '/foo')
self.assertEqual(Cookie.normalize_url_path('/Foo/bar'), '/foo')
self.assertEqual(Cookie.normalize_url_path('/foo/baR/'), '/foo/bar')
def test_normalization(self):
cookie = Cookie('color', 'blue', expires=self.expires)
cookie.timestamp = self.now_timestamp
self.assertEqual(cookie.domain, None)
self.assertEqual(cookie.path, None)
url = 'http://example.COM/foo'
cookie.normalize(url)
self.assertEqual(cookie.domain, 'example.com')
self.assertEqual(cookie.path, '/')
self.assertEqual(cookie.expires, self.expires)
cookie = Cookie('color', 'blue', max_age=self.max_age)
cookie.timestamp = self.now_timestamp
self.assertEqual(cookie.domain, None)
self.assertEqual(cookie.path, None)
url = 'http://example.com/foo/'
cookie.normalize(url)
self.assertEqual(cookie.domain, 'example.com')
self.assertEqual(cookie.path, '/foo')
self.assertEqual(cookie.expires, self.age_expiration)
cookie = Cookie('color', 'blue')
url = 'http://example.com/foo'
cookie.normalize(url)
self.assertEqual(cookie.domain, 'example.com')
self.assertEqual(cookie.path, '/')
cookie = Cookie('color', 'blue')
url = 'http://example.com/foo/bar'
cookie.normalize(url)
self.assertEqual(cookie.domain, 'example.com')
self.assertEqual(cookie.path, '/foo')
cookie = Cookie('color', 'blue')
url = 'http://example.com/foo/bar/'
cookie.normalize(url)
self.assertEqual(cookie.domain, 'example.com')
self.assertEqual(cookie.path, '/foo/bar')
#-------------------------------------------------------------------------------
if __name__ == '__main__':
unittest.main()
Reference in New Issue Copy Permalink