Single Sign-On Security: Security Analysis Of Real-Life OpenID Connect Implementations

Posted on 7:45 PM by Tina

This is a guest blogpost by Lauritz Holtmann. He wrote his master thesis:

"Single Sign-On Security: Security Analysis of real-life OpenID Connect Implementations"

Lauritz summarizes his exciting results in the following. The thesis was supervised by Vladislav Mladenov, Christian Mainka, and Jörg Schwenk. You can read find his full thesis here.

OpenID Connect 1.0 and OAuth 2.0 are the Single Sign-On Protocols that are implemented in modern web applications. In this post, we outline common issue patterns that were discovered in popular OpenID Connect implementations, give concrete examples of vulnerabilities, and give recommendations for adjustments to the OpenID Connect specification.

Table of Contents

  1. Introduction
  2. Attacker Models
  3. High-Level Setup and Selection of Implementations
  4. Derived Issue Patterns
  5. Derived modifications to the OpenID Connect Security Considerations
  6. References

Introduction

This post outlines the lessons learned during the research on the security of real-life OpenID Connect implementations that were roughly performed between April and October 2020. We presume basic knowledge of OpenID Connect 1.0 and OAuth 2.0 for this post.

First, we describe the attacker models that will be applied in this post. Afterward, we outline a high-level overview of the selection criteria for real-life OpenID Connect implementations. Furthermore, we describe patterns that were regularly observed during the analysis of real-life OpenID Connect Service Provider and Identity Provider implementations. Finally, proposals for adjusted security considerations regarding the OpenID Connect specification are derived.

Attacker Models

In the following, multiple different attacker models are considered. Each individual observation will be categorized into an attacker model that is described hereinafter.

Web Attacker

The web attacker model was introduced by Barth et al. in 2008 [13].

Victim

The victim in this attacker model is either an End-User, a benign Service Provider or a benign OpenID Provider.

Objectives of the Attacker

Depending on the victim, the attacker's objectives may be different. If a web attacker targets an End-User, the ultimate goal is to break the authentication and access restricted resources. Beside this, a malicious actor's objective can be to influence the End-User's session, for instance, to log in or log out a victim without indication to the user. If a web attacker targets a Service Provider or Identity Provider, the primary objective is also an authentication bypass in order to access restricted resources. Additionally, causing notable harm to the application or the underlying infrastructure of the Identity Provider or Service Provider can be the objective of a malicious actor.

Capabilities of the Attacker

The web attacker is the least privileged attacker that is considered in the following. A web attacker is not able to intercept or eavesdrop communication that is not intended for him. If the victim is an End-User, we assume that the victim visits an attacker-controlled website. Additionally, the web attacker may send and receive HTTP requests and provide a trusted Transport Layer Security (TLS) certificate for a controlled domain, for example, attacker.com, to enable encrypted communication with Service Providers and Identity Providers. If explicitly outlined, the web attacker is in charge of a user account at the Identity Provider or the Service Provider.

Malicious Identity Provider

The malicious Identity Provider attacker model was introduced in 2016 by Mainka, Mladenov, and Schwenk [10]. They highlighted that the Identity Provider in traditional Single Sign-On setups like Kerberos is a trusted party. If customers can configure own Identity Providers or Identity Providers are dynamically discovered, these entities must be considered as third parties and must be treated accordingly.

Victim

A malicious Identity Provider could either target an End-User or a benign Service Provider as victim.

Objectives of the Attacker

If the malicious Identity Provider targets an End-User, the attacker's objective is to bypass the authentication of the End-User that used an honest Identity Provider to register at a Service Provider, in order to authenticate as the victim user. If a malicious Identity Provider directly targets a benign Service Provider, beside the authentication bypass, causing harm to the underlying application or infrastructure can be the objective of a malicious actor.

Capabilities of the Attacker

A malicious Identity Provider is able to perform a genuine OpenID Connect flow and to authenticate users for a Relying Party. In doing so, it is not bound to restrictions being made by the specification and could also act maliciously. For instance, the malicious Identity Provider may send malformed requests, set id_token claims in a misleading manner or may tamper with the key references that are used for signature validation. A prerequisite for this is that either a victim Client supports Dynamic OpenID Provider Discovery, for example, using the WebFinger protocol [16] or a manually configured and initially trusted IdentityProvider turns rogue at some point in time. In general, as per the overall setup of Single Sign-On, the Identity Provider is a third party from a Client's perspective

Malicious Service Provider

Victim

A malicious Service Provider could either target an End-User or a benign Identity Provider as a victim.

Objectives of the Attacker

If a malicious Service Provider targets an End-User, his main objective is to gather sensitive information about the End-User and to gain privileges to perform actions on behalf of the End-User. In contrast, if a benign Identity Provider is targeted, besides authentication bypasses, the underlying application and infrastructure of the Identity Provider can be the target of the malicious Service Provider.

Capabilities of the Attacker

A malicious Service Provider is able to start a genuine OpenID Connect flow and retrieve a code, an id_token and an access_token from an Identity Provider. Just like the malicious Identity Provider, it may act maliciously when communicating with a victim Identity Provider or victim End-User. In contrast to the regular web attacker, it is in charge of valid client credentials. Thus, a malicious Service Provider needs to initially register at the benign Identity Provider in order to obtain these client credentials. This could either be done manually or using the OpenID Connect Dynamic Client Registration [6].

Malicious Administrative User

The malicious administrative user attacker model was introduced by Matsumoto et al. in 2014 [17, Section 3.1]. The impact of attacks under this attacker model is highly conditional, as described in the following.

Victim

A malicious administrative user can either target a benign Service Provider or a benign Identity Provider as a victim. Additionally, a malicious administrative user could target End-Users of the Provider he has administrative privileges on.

Objectives of the Attacker

The main objective of a malicious administrative user is to target the application or the underlying infrastructure of a Service Provider or Identity Provider. In some cases, a malicious administrative user could additionally target End-Users of the Service Provider or Identity Provider and try to obtain sensitive OpenID Connect related parameters that should not be accessible to administrative users.

Capabilities of the Attacker

A malicious administrative user is a powerful attacker. As he is privileged, he may change OpenID Connect related configurations either at the Identity Provider or at the Service Provider. Thus, exploitability and severity of security issues that occur under this attacker model are highly conditional. Considering a hosted service, a customer that is able to exploit vulnerabilities to target the service directly could in fact cause notable harm. In contrast, if the software is self-hosted, the administrative user is highly trusted in most cases and might have even further access to the system than through a web service. Nevertheless, if the environment enforces the concept of least privileges, there may still be the possibility for malicious administrative users on one host to gain access to restricted resources by exploiting vulnerabilities in the self-hosted software and escalate privileges.

Man-in-the-Middle

The most powerful attacker that is considered in the following is the man-in-the-middle attacker. Man-in-the-middle attacks are "well-known in security environments, and have drawn significant attention" [18].

Victim

In an OpenID Connect Flow, every entity can be the target of a man-in-the-middle attacker, as the attacker operates on the requests and responses that are sent over the network.

Objectives of the Attacker

A malicious entity's main goal is to bypass authentication and authenticate as an End-User at a Service Provider. Additionally, a man-in-the-middle attacker can aim to learn sensitive information and personal data that is transferred over the network.

Capabilities of the Attacker

In the following, we consider that if TLS is used and enforced, all entities use TLS in a secure manner. Thus, the man-in-the-middle attacker is not able to tamper with HTTPS communication, but he is able to obtain, modify, or drop unencrypted communication like plain HTTP without TLS.

High-level Setup and Selection of Implementations

During the analysis, we evaluated OIDC Identity Provider and Service Provider implementations regarding a defined set of assertions, based on the specification's security considerations [2, Section 16][3] and the OAuth 2.0 Security Best Current Practices [1]. In doing so, we analyzed self-hosted software and hosted services. For Service Providers, we considered the ability to configure custom Identity Providers as mandatory.

The following software and services were analyzed:

Derived Issue Patterns

In the following, we outline seven issue-patterns that we derived within our test-set.

Pattern 1: Redirect URI Schemes

The OpenID Connect Core specification is quite vague regarding allowed schemes for redirect_uri values, see [2, Section 3.1.2.1]:

REQUIRED. Redirection URI to which the response will be sent. This URI MUST exactly match one of the Redirection URI values for the Client pre-registered at the OpenID Provider, with the matching performed as described in Section 6.2.1 of [RFC3986] (Simple String Comparison). When using this flow, the Redirection URI SHOULD use the https scheme; however, it MAY use the http scheme, provided that the Client Type is confidential, as defined in Section 2.1 of OAuth 2.0, and provided the OP allows the use of http Redirection URIs in this case. The Redirection URI MAY use an alternate scheme, such as one that is intended to identify a callback into a native application.

We observed that multiple Identity Providers support pre-registering redirect_uri values with any possible scheme. Such schemes' support is potentially dangerous, as browsers treat risky schemes like data or javascript very differently (some of my test cases can be found here, recently quinn found an interesting vector for Firefox using the ws scheme).
Additionally, the specification defined the redirect_uri vague because it had native clients in mind. There are very few legitimate use-cases for these schemes as redirect_uri values in OpenID Connect setups.

Example: Keycloak (CVE-2020-10776)

Keycloak is an open-source Identity and Access Management Software (IAM) maintained by Red Hat. In addition, Keycloak is used as the base for Red Hat's Red Hat Single Sign-On. The following issue was resolved with Keycloak 11.0.3.

Attacker Model

A malicious administrative user or a malicious Service Provider that registers using OpenID Connect Dynamic Client Registration are considered as attacker models in the following.

Description

According to the OpenID Connect specification, alternative schemes may be used as a redirect_uri besides https. If the Client Type is configured as confidential, http is allowed as a scheme. For native applications – e.g. the Twitter Application – even custom schemes like twitter:// may be used [2, Section 3.1.2.1.]:

REQUIRED. Redirection URI to which the response will be sent. This URI MUST exactly match one of the Redirection URI values for the Client pre-registered at the OpenID Provider, with the matching performed as described in Section 6.2.1 of [RFC3986] (Simple String Comparison). When using this flow, the Redirection URI SHOULD use the https scheme; however, it MAY use the http scheme, provided that the Client Type is confidential, as defined in Section 2.1 of OAuth 2.0, and provided the OP allows the use of http Redirection URIs in this case. The Redirection URI MAY use an alternate scheme, such as one that is intended to identify a callback into a native application.

Following the specification consequently, Keycloak did not restrict the redirect_uri regarding its scheme at all. This miss enabled dangerous behavior, as the Authentication Response's 302 redirect was sent to this URI with an arbitrary scheme. As a result, depending on the Browser's behavior and treatment of the Location HTTP header, this could cause several security issues, potentially even leading to Cross-Site-Scripting (XSS).

Browser behavior regarding dangerous schemes in Location headers highly differs and changed in the last decade. Among a test-set of current and popular browser versions, only Safari 13.5 (iOS and macOS) executes JavaScript provided as data-URI, as shown in the following:


The JavaScript execution would be limited, as the script is executed in a different origin than the Identity Provider that sets the Location header (null origin). Thus, cross-origin access was restricted by the Same-Origin Policy. Nevertheless, an attacker may be able to launch a Cross-Origin State Inference (COSI) attack like Sudhodanan et al. pointed out in their paper "Cross-Origin State Inference (COSI) Attacks: Leaking Web Site States through XS-Leaks" [12] using a data-URL as redirect_uri.

Steps to reproduce

The following steps apply for Keycloak versions < 11.0.3:

  1. Register a new client and configure the following recirect_uri: data:text/html,%3Cscript%3Ealert%281%29%3C%2Fscript%3E 

  2. Visit the Authentication Endpoint of the created Client: https://keycloak.local:8443/auth/realms/master/protocol/openid-connect/auth?scope=openid&state=a&response_type=code&client_id=test&redirect_uri=data%3Atext%2Fhtml%2C%253Cscript%253Ealert%25281%2529%253C%252Fscript%253E&nonce=b
  3. After successful authentication, there is a redirect to the registered redirect_uri:
HTTP/1.1 302 Found Location: data:text/html,%3Cscript%3Ealert%281%29%3C%2Fscript%3E?           state=a&session_state=b&code=c [...]

Depending on the Browser's behavior, the data-URI is parsed and the embedded JavaScript is executed. For instance, Safari 13.5 evaluates the provided Location header, as presented earlier.

Responsible Disclosure

Pattern 2: Server-Side-Request-Forgery

Multiple Identity and Service Provider implementations were vulnerable to different types of Server-Side Request Forgery (SSRF) vulnerabilities. The Identity Provider's request_uri parameter is already known to be vulnerable for SSRF by design since 2017 [4, Section III;A. 8)]. Notably, this vulnerability could be exploited by any unauthenticated user.

In 2017, Mladenov and Mainka pointed out [5, Section 2.1.2] that a malicious Discovery Service could be used to launch an SSRF attack against a Service Provider. Thus, the attacker can use SSRF for port scans or for retrieving data. Additionally, having access to the endpoint configuration, administrative users could launch SSRF attacks on the Service Provider by design. Depending on the actual setup, this yields to severe security implications, especially considering hosted and cloud services.

Even though these issues were theoretically discussed earlier, SSRF to private IPs or localhost was a common and serious issue among our test-set.

Example 1: Unauthenticated SSRF in Keycloak (CVE-2020-10770)

Keycloak is an open-source Identity and Access Management Software (IAM) maintained by Red Hat. In addition, Keycloak is used as a base for Red Hat's Red Hat Single Sign-On.

Attacker Model

The attacker model applied for the following attack is an unauthenticated web attacker model introduced by Barth et al. in 2008 [13]. In the following, the malicious entity targets a benign Identity Provider. The attacker's objective is to gain access to internal hosts that are situated within the internal network; direct access to these hosts is restricted using a firewall.

Description

The request_uri is an optional parameter within the OpenID Connect Authentication Request that allows specifying an external URI where the Request Object can be found. Fett et al. discovered in 2017 [14, Section III;A. 8)] that, as the Identity Provider is supposed to request the external Request Object, this parameter can easily launch an SSRF attack against the Identity Provider.

The OpenID Connect Core specification defines the request_uri as an Authentication Request parameter that "[...] enables OpenID Connect requests to be passed by reference, rather than by value" [14, Section 6.]. If a Service Provider uses this parameter, the Identity Provider retrieves the Request Object "[...] from the resource at the specified URL" [2, Section 6.2.].

The Identity Provider could restrict allowed URLs by allowing the Service Provider to specify the request_uris parameter that is an "[...] array of request_uri values [...]", within OpenID Connect Dynamic Client Registration [6, Section 2.]. If the OpenID Connect Dynamic Client Registration is used, the Identity Provider can require the Service Provider to "[...] pre-register request_uri values using the request_uris parameter [...]" [2, Section 6.2.]. Thus, only if OpenID Connect is implemented with the Registration Extension, there is a mitigation hint given by the specification.

Keycloak supported using a Request Object and referencing it externally. The "Fine Grained OpenID Connect Configuration" for a specific client allowed to specify the following values for the option "Request Object Required":

  • Not required (default)
  • request or request_uri
  • request only
  • request_uri only



As one could observe, if an OpenID Connect Client was set-up with default settings, the default value for this option was "Not required". There was no option to do not support a Request Object at all, as "not required" meant submitting the request object is optional, but if you submitted one, it was used by Keycloak. Additionally, there was no option to manually define the request_uris parameter during manual client registration. Finally, using the default configuration, Keycloak did not require a Request Object, but supported passing the request_uri parameter with an Authentication Request. As a result, Keycloak was vulnerable to the SSRF attack Fett et al. described in 2017 [14, Section III;A. 8)] in its default configuration for Clients.

Proof-of-Concept

A straight forward PoC to trigger the SSRF issue is given with the following URL (Authentication Response): https://keycloak.local/auth/realms/master/protocol/openid-connect/auth?scope=openid&response_type=code&redirect_uri=[Valid-RedirectURI]&state=aaaa&nonce=bbbb&client_id=[Valid-ClientID]&request_uri=http://127.0.0.1:1234

Thus, to launch the SSRF in a real-life scenario, an attacker would obtain an Authentication Request (to get the valid client_id and redirect_uri).

If the attacker then sends the Authentication Response GET request including valid client_id and redirect_uri, Keycloak sends a GET request to 127.0.0.1:1234 to fetch the Request Object resulting in blind SSRF.

By measuring the response time, a malicious actor may additionally perform a port scan of localhost or internally accessible hosts. The following ports are open on localhost:

  • 8080: Open, Keycloak, 1087 Bytes as default response
  • 8081: Open, Mailslurper, 4086 Bytes as default response
  • 5555: Closed
  • 5556: Closed

Demo: An example using the Timeinator Burp Plugin [15] is presented below: 


In our test, Keycloak responded significantly faster if the port was closed. Thus, the Blind SSRF enabled an attacker to learn about the internal network structure.

Responsible Disclosure
  • 2020-04-29: Initial Report to Red Hat via https://issues.redhat.com/projects/KEYCLOAK/.
  • 2020-06-05: Red Hat triages the submitted report.
  • 2020-06-11: CVE-2020-10770 is assigned.
  • 2020-10-16: Red Hat is informed that this post is scheduled for 2020-11-10.
  • 2020-10-19: Red Hat approves to disclosed this post on the described vulnerability:

Thank you for informing about this. Since this is a Moderate impact flaw, we have 365 calendar days to publish the fix after the issue has been made public so should not be a problem publishing it on 2020-11-10.

- Internal Comment by Red Hat Employee

According to the internal ticket status, Red Hat planned to resolve this issue with the upcoming major version v12 of Keycloak.

Example 2: SSRF in Amazon Cognito (AWS)

Likewise Keycloak, Amazon Cognito implements Identity and Access Management (IAM). But in contrast, as part of the AWS family, Amazon Cognito is a hosted service instead of a self-hosted software.

Attacker Model

The attacker model that is applied in the following is a malicious administrative user. The impact of attacks under this attacker model is highly conditional, but in this case, as Amazon Cognito is a hosted service, administrative users only have access to the configuration interface. The underlying infrastructure in Amazon's hosted environment should not be accessible to external users.

Alternatively, a malicious Identity Provider could be considered as the attacker model. The actual configuration is fetched using OpenID Connect Discovery from the Identity Provider's Configuration Endpoint so that the endpoints are Identity-Provider-controlled.

Description

Amazon Cognito allowed specifying custom OpenID Connect Identity Providers for user pools. Within the configuration of an Identity Provider, an administrative user could choose to "Run discovery", triggering an OpenID Connect Discovery that requests the Configuration Endpoint of the Identity Provider. Amazon Cognito did not restrict the endpoints observed using OpenID Connect Discovery regarding internal IP addresses or localhost so that SSRF to the local network and localhost was possible.

A malicious actor could utilize this to perform port scans or send nearly arbitrary HTTP requests to hosts that are intentionally not exposed to the internet. An example Configuration Response is presented in the following:

{     "issuer":"https://example.com/",     "authorization_endpoint":"https://example.com/auth",     "token_endpoint":"http://127.0.0.1:22",     "userinfo_endpoint":"http://127.0.0.1:22",     "jwks_uri":"https://example.com/jwks",     "registration_endpoint":"https://example.com/register",     "response_types_supported":["code","token id_token"],     "subject_types_supported":["public","pairwise"],     "id_token_signing_alg_values_supported":["RS256"] }

Portscan

The following error messages could be used to determine which ports are internally open on localhost, as the error message sent to the redirect_uri (https://a.com/cb?error_description=[ERROR]&error=invalid_request) differed. Depending on the underlying Transmission Control Protocol (TCP) connection (if it cannot be established) and the HTTP error code Amazon Cognito received in response to the Token Request, the following error messages were used:

  • If the Token Endpoint was specified as "http://localhost:22", the error message was "Connection reset", so that we could assume Port 22 to be open.
  • If the Token Endpoint was specified as "http://localhost:20", the error message was "Connect to 127.0.0.1:20 [/127.0.0.1] failed: Connection refused", so that we could assume Port 20 to be closed.
  • If the Token Endpoint was specified as "http://test123.ngrok.io", the error message was "test Error – 502 error getting token", so that we could assume that there was an HTTP request to the specified target, but the webserver responded with HTTP Error Code 502.

Arbitrary HTTP requests

Besides the feedback an attacker receives that enabled him to determine the connection's status, there was no information on the actual response to the HTTP request. Thus, this gadget could be considered as blind SSRF to the local network. The attacker had no direct feedback but could still control a GET request (UserInfo Request) and a POST request (Token Request) regarding scheme (http or https), host (potentially localhost or internal IP), path and query parameters.

Combining port scan and blind SSRF, a malicious administrative user or malicious Identity Provider could:

  1. Gather information: Which ports are open on localhost? Are there other hosts accessible? If HTTP error codes are reflected, which web service is running at this destination (fingerprinting)?
  2. If a service could be identified: Use blind SSRF to target internally accessible service directly.

Recommendation

A blacklist is never perfect. Nevertheless, Amazon Cognito should introduce a restriction for internal IPs and localhost as OpenID Connect endpoints, as there is no legitimate use-case in the context of a hosted service.

The fix that was applied in late August 2020 mitigates the above described vulnerability.

Responsible Disclosure

Mitigations for the described issues were introduced by AWS / Amazon Cognito soon after we reported the issue on 2020-08-12.

  • 2020-08-12: Initial report to aws-security@amazon.com.
  • 2020-08-19: Call with AWS Security Team and Amazon Cognito Developers.
  • 2020-08-??: A fix is applied at the end of August 2020. As Amazon Cognito is a hosted service, we could not determine the exact date.
  • 2020-10-21: AWS Security Team acknowledges that the blog post is scheduled for 2020-11-10 and offers feedback for the draft prior to publication:

Thanks again for bringing your security concern to our attention. We greatly appreciate and encourage reports from the security community.

I understand you wish to do a write up on this - would you be interested in sharing your draft with us before publishing so we may provide any assistance and feedback?

  • 2020-11-05: After providing an initial draft, the AWS Security team would like to explicitly outline that the described vulnerability was mitigated shortly after we reported it responsibly.

Pattern 3: TLS Enforcement

The OpenID Connect Core specification already enforces TLS for communication with Authentication, Token, and UserInfo Endpoint, so does the OpenID Connect Dynamic Client Registration regarding the Client Configuration Endpoint. Nevertheless, most of the analyzed implementations allowed performing the discovery using unencrypted communication or allowed to specify OpenID Connect endpoints using HTTP without TLS.

Pattern 4: Leakage via Error Pages

Error messages and pages need extra attention. We discovered that multiple Service Provider implementations presented error pages as a direct response to Authentication Responses [2, Section 3.1.2.5.]. As a result, this page's GET parameters include OpenID Connect values like state and code. External resources like images, scripts, styles, or links to external resources can be on any endpoint. In these cases, the attacker can retrieve sensitive information, including OpenID Connect parameters, in query parameters or the Referer header.

Example: Bitbucket Server

Bitbucket Server is a self-hosted Git management software: https://www.atlassian.com/software/bitbucket

Attacker Model

Even though the following section describes a flaw that leads to the disclosure of sensitive information to third parties, there is no actual active attacker present. Due to the overall setup as a self-hosted service, Atlassian normally does not receive any information about ongoing OpenID Connect authentication processes. Nevertheless, the below-presented flaw leads to disclosure of sensitive OpenID Connect parameters to Atlassian.

Description

During authentication using a custom OpenID Connect Identity Provider, Bitbucket returned an error after the End-User was sent to the redirect_uri in case the End-User's identifier (i.e., "sub" claim of the OIDC id_token) could not be found in Bitbucket's user base. The following error was written to the application log:

2020-05-04 13:29:44,644 ERROR [http-nio-7990-exec-4] @1GWJER3x809x186x0 16qfs24      172.17.0.1,172.17.0.1 "GET /plugins/servlet/oidc/callback HTTP/1.1"      c.a.p.a.i.w.f.ErrorHandlingFilter Received SSO request for user      [sub value], but the user doesn't exist in the product

As a result, the user received a generic error message that was displayed as a direct response to the OpenID Connect Authentication Response redirect so that the URL still included sensitive OpenID Connect parameters like code and state: https://bitbucket.local/plugins/servlet/oidc/callback?state=[RE-DACTED]&session_state=[REDACTED]&code=[REDACTED]. The code value is very valuable, as it is used by the Service Provider to obtain the access_token and id_token from the Identity Provider. Furthermore, the error page included references to external resources. Bitbucket Server is a self-hosted software so that Atlassian can be considered as a third party in this case and should not receive any OpenID Connect parameters. As there was no Referrer-Policy present on this error page, the full path of the referring website including OpenID Connect state and code was disclosed to third parties if a user clicked one of the links to external resources, as shown in the following:


<ul> 	<li data-key="footer.license.free.eval"> [...]          <a href="http://www.atlassian.com/software/bitbucket/">          Atlassian Bitbucket         </a> evaluation license</li> </ul><ul> [...] 	<li data-key="footer.links.documentation">         <a href="http://docs.atlassian.com/bitbucketserver/ docs-072/          Bitbucket+Server+documentation?utm_campaign=in-app-help          &amp;utm_medium=in-app-help&amp;utm_source=stash" target="_blank">         Documentation</a></li> 	<li data-key="footer.links.jac">         <a href="https://jira.atlassian.com/browse/BSERV" target="_blank">         Request a feature</a></li>         [...] </ul>
Responsible Disclosure
  • 2020-05-04: Initial Report via https://bugcrowd.com/atlassian
  • 2020-05-07: The status of the report is set to triaged by a Bugrowd employee.
  • 2020-05-08: Atlassian asks for further information and clarification on the potential impact.
  • 2020-05-15: With the additional information that was provided, Atlassian decides that the reported behavior will be addressed.

Thanks for your detailed response. It's our opinion that due to the very high complexity of this attack, and the niche circumstances required it is of a limited impact, but this is an issue we'll be aiming to fix.

  • 2020-11-16 The status of the reported issue is set to resolved.

Pattern 5: CSRF

The Service Provider's Initiate Login Endpoint implements Login Cross-Site-Request-Forgery (CSRF) by design and per specification [2, Section 4.]. This behavior is well known and was previously discussed [4, Section III; A. 7)]. Besides this endpoint, we observed non-normative endpoints that start an external OpenID Connect login without CSRF protection as well. If the state is correctly validated, these endpoints still allow to log in a victim End-User into her own account ("Login CSRF").

If there are additional non-normative parameters that indicate where the End-User should be redirected to after successful authentication (i.e., "next", "nextUrl", ...) and these parameters allow specifying the Login Initiation Endpoint or non-normative endpoints that start external OIDC login flows, Login Confusion attacks are possible. Technical details for this novel attack are presented in this post.

Example: Bitbucket Server

Bitbucket Server is a self-hosted Git management software: https://www.atlassian.com/software/bitbucket

Setup and Prerequisites

For our test environment, we used the official Docker image available at https://hub.docker.com/r/atlassian/bitbucket-server/. Additionally, we needed to set up our instance with a "Data Center" license, as the required "SSO 2.0" configuration would not be available in case we use a "Server" license. Finally, we require that the Bitbucket instance has a (secondary) OpenID Connect Identity Provider configured (Bitbucket can be configured to either only support authentication using OIDC or using OIDC as secondary authenticator with credentials as primary alternative method).

For our victim, we require that the victim browses an attacker-controlled website. Prerequisites for this attack are that the victim has two accounts at Bitbucket. For user A our victim uses credentials. User B is authenticated using an external Login Provider. Additionally, the victim has an active session at the Identity Provider.

Steps to Reproduce

Bitbucket implements two endpoints that enable intentional login CSRF. Both endpoints did not implement CSRF mitigations so that with a request to https://bitbucket.test/plugins/servlet/oidc/initiate-login or https://bitbucket.test/plugins/servlet/external-login an OpenID Connect login flow was started. If a user had an active session at the Identity Provider and previously gave consent to share information with Bitbucket as Service Provider, the Identity Provider immediately responded with the Authentication Response when the Authentication Endpoint is queried.

Furthermore, both endpoints allowed GET parameters that specified where the user should be redirected to after successful authentication. While this parameter was sanitized to prevent Covert Redirects to external destinations, any path of the Bitbucket instance was allowed as a final destination, including session-related endpoints like the Logout Endpoint (that terminates the user's session) or the Single Sign-On Login Initiation Endpoint. This sanitization enabled the following Login Confusion attack: 


Prerequisite (recall): The victim has an active session at the Identity Provider and previously used this account to authenticate as user B at Bitbucket.

  1. The victim visits an attacker-controlled website and is redirected to https://bitbucket.test/login?next=/plugins/servlet/external-login.
  2. Bitbucket serves a regular Login Form, the victim authenticates using her credentials for user A (1).
  3. After successful authentication, Bitbucket redirects the victim to the Login Initialization Endpoint, as this was the previously provided destination (2).
  4. Bitbucket receives the GET request to the Login Initiation Endpoint and launches a new OpenID Connect Login flow accordingly.
  5. The victim has, per our prerequisites, an active session at the Identity Provider. As a result, there is a silent redirect with the Authentication Response to the redirect_uri endpoint including valid code and state parameters.
  6. Bitbucket validates the state and redeems the code at the Identity Provider's Token Endpoint. The Identity Provider responds with an id_token and access_token.
  7. After validating the identity of user B, the victim is logged in as user B, although she entered her credentials for user A and did not perform any additional interactions.

To conclude the previous steps: The victim authenticates using credentials for user A but is then logged in as user B after multiple intransparent redirects and without further user interaction. This underlines the conclusion Fett et al. [11, Section III;A. 7)] drew in 2017: Endpoints that provide OpenID Connect related functionality require Cross-Site-Request-Forgery protection.

Demo

The following video demonstrates the Login Confusion issue present in Bitbucket. 



Note that in the above video the native account is userB and the SSO account is userA.


Impact

The above-described behavior has a direct effect on the End-User's session, does not require any privileges, and leads to an unforeseen login status from the End-User's perspective. If an End-User does not observe that the finally authenticated account is not the intended account, the End-User may perform actions with the wrong account that could, for example, leak their SSO Identity to third parties.

The impact is limited by the fact, that both accounts are under the control of the victim. The attacker cannot log in a victim into an attacker-controlled account, unless the Identity Provider has further flaws.

Responsible Disclosure

Atlassian's triage process on Bugcrowd is very transparent. Their Security Team outlined that the above-described behavior is "intended behavior". The report's status was changed to "won't fix" accordingly.

During direct communication with Atlassian's security team using mail, they allowed using Bitbucket as an example in this blog post.

  • 2020-06-08: Submission of the described behavior as "follow-up" to initially rejected report via https://bugcrowd.com/atlassian.
  • 2020-08-18: Atlassian's Security Team informs us that they reviewed the follow-up but still consider the report as "not applicable" regarding their policy:

I'm still not able to find the vulnerable component in the workflow. The scenario you described can happen with any IdP, this is not specific to bitbucket.

  • 2020-09-21: Security team agrees on publishing this blog post:

Yes, please feel free to discuss those findings in your blog post. As you mentioned (and as was discussed in those submissions), we've determined each of those issues fall outside of our security model for Bitbucket Data Center and therefore will not be addressed.

Note that not every Service Provider is vulnerable to Login Confusion and that the countermeasures are quite trivial to implement, as presented within this post.

Pattern 6: State parameter handling

OpenID Connect state parameter handling at the Service Provider's Redirection Endpoint showed multiple more common issues. Some of the analyzed implementations did not handle the unexpected behavior of not receiving a state within the Authentication Response if it was present within the Authentication Request. As a result, unhandled exceptions occurred. Depending on the actual implementation, this may lead to sensitive information disclosure or availability issues. Further, multiple implementations correctly bound the state to the user session. Still, they failed to make it one-time-usable, increasing the attack surface for token-reuse and Denial-of-Service Amplification attacks.

A novel real-life attack scenario if the state is not invalidated is the DoS Amplification attack. Note that this attack's impact is conditional. For example, this vulnerability has a potentially higher impact in a hosted service than in a self-hosted environment.

Suppose the Service Provider allows reusing the state parameter and does not implement an additional rate-limiting at the redirect_uri endpoint. In that case, it can be exploited as an "amplifying proxy" for Denial-of-Service attacks. This is in some-regards comparable to previously known DNS Amplification Attacks. It has been observed that the outgoing Token Request (back-channel) to the configured Token Endpoint can be multiple times larger than the Authentication Request (front-channel). A malicious actor could utilize this to launch a Denial-of-Service attack spending limited bandwidth using the Service Provider as an amplifying proxy, as presented in the following: 


An unauthenticated web attacker could only target the configured Identity Provider. A malicious Identity Provider can additionally specify an arbitrary web server as Token Endpoint with a long URL path and issue long client credentials on registration (either manual or using OpenID Connect Dynamic Client Registration), resulting in an even larger outgoing request.

If there is no rate limiting at the redirect_uri endpoint, the attacker could continuously replay the Authentication Response including the valid state, resulting in large outgoing Token Requests on each try.

Example: Keycloak (CVE-2020-14302)

Keycloak is an open-source Identity and Access Management Software (IAM) maintained by Red Hat. In addition, Keycloak is used as the base for Red Hat's Red Hat Single Sign-On.

Attacker Model

Two attacker models could be considered (three if you also consider the high privileged malicious administrative user). A web attacker having no influence on the target of the Token Request could only target the already configured Identity Provider. A malicious Identity Provider using the OpenID Connect Discovery could additionally target arbitrary web servers accessible for the Service Provider.

Description

Keycloak as Service Provider provides dedicated redirect_uris for each Identity Provider: https://keycloak.local/auth/realms/{realm}/broker/{alias}/endpoint.

The redirect_uri endpoint did not invalidate state values if they were redeemed once. As a result, multiple requests to this endpoint including a valid state value resulted in requests to the configured Token Endpoint being initiated by Keycloak. A malicious administrative user or malicious Identity Provider could additionally increase the outgoing Token Request by choosing a looong path for the Token Endpoint URL and setting imaginary looong Client Credentials. As a result, one request to the Keycloak instance could be amplified multiple times compared to the request that is received at the victim server's end.

Steps to reproduce

The steps to launch a Denial-of-Service attack using many large requests through Keycloak as amplifying proxy are the following:

  1. Configure the victim server as Token Endpoint with a long path and long Client Credentials (if the attacker model is a regular web attacker, this step is skipped).
  2. Start an OpenID Connect flow and obtain a valid state value (i.e., save a valid Authentication Response including the state).
  3. After the Authentication Response is received, Keycloak tries to redeem the received code value at the victim web server. This request can be multiple times larger than the attacker's request (Authentication Response).
  4. Replay the obtained Authentication Response, on each try Keycloak sends a new HTTP request to the victim server.

Responsible Disclosure

  • 2020-06-16: Initial report to Red Hat via https://issues.redhat.com/projects/KEYCLOAK/.
  • 2020-06-22: CVE-2020-14302 is assigned.
  • 2020-10-20: Red Hat is informed that the publication of this post is scheduled for 2020-11-17.

Pattern 7: Injection of Metacharacters

Service Providers often treat contents provided by Identity Providers as trustworthy. In our research, we show that this behavior in general yields injection issues (previously outlined as "Injection Attacks" by Mainka et al. in [10, Section 2.1.3]). Striking among the test-set were CRLF-related issues, as missing sanitization can lead to HTTP header injections in Service Provider initiated requests and injections to application log files. Specification-wise these issues should be already addressed, as RFC6750 specifies the allowed character-set for bearer credentials in the context of HTTP header [8]. OIDC implementations must implement this as whitelist - excluded characters must be stripped from the access_token before it is used in sensitive contexts:

The syntax for Bearer credentials is as follows:

b64token = 1*( ALPHA / DIGIT / "-" / "." / "_" / "~" / "+" / "/" ) "=" credentials = "Bearer" 1SP b64token

Example 1: GitLab - Request Splitting

Suppose GitLab is configured to use an external OpenID Connect login provider. In that case, it obtains the access_token from the Identity Provider's Token Endpoint and redeems this token as bearer token when requesting the Identity Provider's UserInfo Endpoint. In doing so, GitLab did not correctly sanitize the token value for usage in the context of HTTP headers.

As a result, a malicious Identity Provider could inject CRLF sequences into the access_token that were then directly injected into the request performed by GitLab, resulting in HTTP header injection and HTTP request splitting. Additionally, the malicious Identity Provider could then arbitrarily choose the target of the UserInfo Request, without restrictions regarding localhost or private IPs within the Configuration Response. The malicious Identity Provider could then send arbitrary HTTP requests (verb, headers, body) to any internal hosts, external hosts, or localhost.

Recap of the generic attack flow as presented earlier: 


Note that the target of the UserInfo Request was not altered (for illustrative purposes) but could point to an arbitrary host.

Suppose a Redis instance is present either on localhost or within the internal network. In that case, this attack's impact can even be escalated to Remote Code Execution, as it was previously shown for other SSRF vulnerabilities with the ability to add HTTP headers in the context of GitLab [9].

Responsible Disclosure

The users who can configure the identity providers are administrative users and are in a high level of trust within a company. Thus, there do not appear to be any security implications as a direct result of this behavior that involves rogue IdPs.

  • 2020-07-03: GitLab closes the report as "informative".
Example 2: Bitbucket - Log Injection

If Bitbucket uses an OIDC "SSO 2.0" authentication provider and a custom "Username claim" is specified, Bitbucket queries the UserInfo Endpoint using the previously obtained access_token. In doing so, Bitbucket failed to properly sanitize the bearer token before setting the HTTP header to its value. As a result, if the token included a CRLF sequence, an unhandled exception occurred in sun.net.www.protocol.http.HttpURLConnection.checkMessageHeader caused by illegal characters in the context of HTTP headers. This exception, including the access_token was written in plain text and without stripping sensitive information or dangerous characters to the application log file situated at /var/atlassian/-application-data/-bitbucket/log/atlassian-bitbucket.log.

An example is shown below:

2020-07-09 21:31:23,098 ERROR [http-nio-7990-exec-10]  o.a.c.c.C.[.[.[/].[plugins]      Servlet.service() for servlet [plugins] in context with path [...]      threw exception java.lang.IllegalArgumentException: Illegal character(s) in message header      value: Bearer AccessTok[HERE IS THE CRLF SEQUENCE] en 	at sun.net.www.protocol.http.HttpURLConnection      .checkMessageHeader(HttpURLConnection.java:542) [...]

Besides the storage of sensitive information in log files, this yielded additional security risks. Bitbucket comes with a GUI log analyzer that parses the application log. Using the log injection, it was possible to spoof log entries that are rendered and presented to administrative users.

As the log analyzer uses regular expressions to search for known issues and only renders a trusted description and markup (no attacker-controlled contents), no further injection issues within the frontend representation of injected log entries could be observed. Nevertheless, if the log analyzer was configured to run periodically and sends mails on recognized errors, a malicious actor could trick Bitbucket to send notification emails to administrative users.

Responsible Disclosure
  • 2020-06-10: Initial submission via https://bugcrowd.com/atlassian.
  • 2020-06-17: Report passes triage and is forwarded to Atlassian.
  • 2020-06-28: Atlassian closes the report as "Not Applicable"

We have deemed that this report is not applicable. Since an attacker would need to have administrator permission to configure malicious IdP which is higher level of privileges which achieved by this attack.

Derived modifications to the OpenID Connect Security Considerations

In addition to previously known security considerations [1][2, Section 16.][5], it has been shown that the following aspects need extra attention. Some of these considerations have been made earlier but with a less severe indication. In terms of RFCs and specifications, this would mean that a "MAY" or "SHOULD" would become a "MUST".

  1. The specification needs to be tightened regarding the redirect_uri definition. This URI scheme should NOT be data, javascript, vbscript, or any other scheme that is not HTTP(S) or related to a dedicated native client.
  2. The Identity Provider SHOULD restrict allowed request_uri values by allowing the Service Provider to specify the request_uris parameter that is an "[...] array of request_uri values [...]", at registration [6, Section 2.]. Currently, only if the OpenID Connect Dynamic Client Registration is used, the Identity Provider can require the Service Provider to "[...] pre-register request_uri values using the request_uris parameter [...]" [2, Section 6.2.]. This registration should be moved from the Registration Extension to the Core Specification.
  3. In the context of OpenID Connect, Identity and Service Providers SHOULD globally restrict outgoing HTTP requests ("backchannel communication") to localhost and private IPs [7, Section 3.]. There MAY be a configurable option to allow these connections which is only accessible for high privileged users. Further, error messages that are returned in case an OpenID Connect request fails MUST NOT include connection-specific details that could give an attacker information on the underlying transport layer (for instance error messages that indicate if a TCP connection was refused or reset).
  4. As the Specification already outlines, for instance in [6, Section 7.2.], OpenID Connect Service and Identity Providers MUST support and MUST use TLS for communication with the Configuration, Token and UserInfo Endpoint.
  5. Error messages SHOULD not be shown as a direct response to erroneous OpenID Connect requests if the error page includes external resources, without omitting the sensitive GET parameters from Referer headers. This could either be done using a prior redirect to a generic path (e.g., https://example.com/error) or by defining a restrictive Referrer-Policy (e.g., "origin-when-cross-origin" or "strict-origin-when-cross-origin").
  6. Service Providers MUST use a state value to prevent CSRF Login into arbitrary accounts if no other CSRF protection is implemented. If the state was present within the Authentication Request, it MUST be included within the Authentication Response. If this is not the case, the Service Provider needs to fail safely (i.e., not ignore that the state is missing and proceed anyway, but handle the occurring exception adequately). In addition, the state value MUST be invalidated after it was redeemed once to prevent amplification attacks.
  7. As previously advised by Fett et al. [4, Section III; A. 7)], the optional Initiate Login Endpoint SHOULD NOT be implemented if it is not explicitly needed. The same applies to non-normative external login endpoints. If these endpoints are implemented, token- or Referer header-based CSRF mitigations MUST be in place. In case there is a non-normative GET parameter that controls the redirection target after successful authentication, session relevant endpoints like the External Login Endpoints or Logout Endpoints SHOULD NOT be allowed as a redirection target.
  8. Contents provided by Identity Providers MUST be handled as third party input. Therefore, strict and context-aware sanitization MUST be performed before using these values.

References

[1] https://tools.ietf.org/html/draft-ietf-oauth-security-topics
[2] https://openid.net/specs/openid-connect-core-1_0.html
[3] https://tools.ietf.org/html/rfc6749
[4] https://publ.sec.uni-stuttgart.de/fettkuestersschmitz-csf-2017.pdf
[5] https://www.nds.ruhr-uni-bochum.de/media/ei/veroeffentlichungen/2017/01/13/OIDCSecurity_1.pdf
[6] https://openid.net/specs/openid-connect-registration-1_0.html
[7] https://tools.ietf.org/html/rfc1597
[8] https://tools.ietf.org/html/rfc6750
[9] https://hackerone.com/reports/299473
[10] https://arxiv.org/abs/1412.1623
[11] https://dl.acm.org/doi/10.1145/2976749.2978385
[12] https://arxiv.org/abs/1908.02204
[13] https://www.adambarth.com/papers/2008/barth-jackson-mitchell.pdf
[14] https://dl.acm.org/doi/10.1145/2976749.2978385
[15] https://github.com/FSecureLABS/timeinator
[16] https://tools.ietf.org/html/rfc7033
[17] https://dl.acm.org/doi/10.1145/2620728.2620750
[18] D. Serpanos and R.J. Lipton: Defense against man-in-the-middle attack in client-server systems. Proceedings. Sixth IEEE Symposium on Computers and Communications, pp. 9–14, 2001

Author of this Post

Lauritz Holtmann.
His master thesis was supervised by Vladislav Mladenov, Christian Mainka, and Jörg Schwenk.

Related news


  1. Tools For Hacker
  2. Blackhat Hacker Tools
  3. Hack Tools Mac
  4. Hacker Tools Apk
  5. Hacker Tools 2019
  6. Pentest Tools Website Vulnerability
  7. Hacking Tools Mac
  8. Hack Tools
  9. Growth Hacker Tools
  10. Nsa Hacker Tools
  11. Hacking Tools For Pc
  12. Hack Tools 2019
  13. Hacker Tools For Windows
  14. Pentest Tools List
  15. Hacker Tools For Windows
  16. Hacker Hardware Tools
  17. Hack Tools Github
  18. Hacking Tools Usb
  19. Hacking Tools 2019
  20. Hack Tools Online
  21. Hack Tools For Windows
  22. Hacking Tools 2019
  23. Hacking Tools Usb
  24. Hacking Tools Kit
  25. Hacking Tools Windows 10
  26. Tools Used For Hacking
  27. Hacking Tools
  28. Hacking Tools Windows 10
  29. Pentest Recon Tools
  30. Hacking Tools Software
  31. Tools For Hacker
  32. Nsa Hack Tools
  33. Growth Hacker Tools
  34. Hacking Apps
  35. Tools 4 Hack
  36. Hacking Tools
  37. Hack Tools 2019
  38. Computer Hacker
  39. Hacking Tools
  40. Hacker Tools Apk Download
  41. Growth Hacker Tools
  42. Pentest Tools Bluekeep
  43. Kik Hack Tools
  44. Hacking Tools For Windows
  45. Hacking Tools For Mac
  46. Hacking Tools Mac
  47. Hacking Tools 2019
  48. Hacking Tools For Windows
  49. Pentest Tools Nmap
  50. Pentest Tools Apk
  51. Pentest Tools Tcp Port Scanner
  52. Best Hacking Tools 2020
  53. Hackers Toolbox
  54. Nsa Hack Tools
  55. Hack Tools For Windows
  56. Hacker Search Tools
  57. Best Hacking Tools 2019
  58. Hacking App
  59. Hacker Tools Free
  60. Pentest Tools Android
  61. Pentest Tools For Mac
  62. Github Hacking Tools
  63. Hacker Tools For Pc
  64. Pentest Tools Kali Linux
  65. Nsa Hacker Tools
  66. Hack Tools For Ubuntu
  67. Pentest Tools Review
  68. Hacker Tools Hardware
  69. Pentest Tools Alternative
  70. Pentest Tools Alternative
  71. Beginner Hacker Tools
  72. Best Hacking Tools 2019
  73. Hacking Tools Online
  74. Pentest Tools Framework
  75. Pentest Tools Linux
  76. Pentest Tools Android
  77. Hack Tools For Games
  78. Hack Website Online Tool
  79. Install Pentest Tools Ubuntu
  80. Hacker Tools Apk
  81. Nsa Hacker Tools
  82. Hack Tool Apk No Root
  83. Pentest Tools Online
  84. Github Hacking Tools
  85. Hacker Tools For Ios
  86. Best Pentesting Tools 2018
  87. Hacker Tools For Mac
  88. Pentest Tools Website Vulnerability
  89. Hack And Tools
  90. Pentest Tools Framework
  91. Hackrf Tools
  92. Hacking Tools And Software
  93. Blackhat Hacker Tools
  94. Pentest Tools Apk
  95. Pentest Tools Url Fuzzer
  96. Pentest Tools Github
  97. Hacker Tools For Ios
  98. Hacking Tools Online
  99. Black Hat Hacker Tools
  100. Underground Hacker Sites
  101. Hacker Security Tools
  102. Hacking App
  103. Pentest Tools Windows
  104. Hacking Tools
  105. Hacker Tools 2020
  106. Top Pentest Tools
  107. Hacker Tool Kit
  108. Pentest Tools Port Scanner
  109. Pentest Tools Apk
  110. Hack Rom Tools
  111. Pentest Tools Port Scanner
  112. Tools Used For Hacking
  113. Pentest Tools For Mac
  114. Hak5 Tools
  115. Pentest Tools
  116. Hacker Tools Free Download
  117. Ethical Hacker Tools
  118. Tools 4 Hack
  119. What Is Hacking Tools
  120. Pentest Tools Github
  121. Pentest Tools Framework
  122. Hacker Tools For Pc
  123. Hack Tools
  124. Game Hacking
  125. Usb Pentest Tools
  126. Pentest Tools Port Scanner
  127. Hacker Tools For Mac
  128. Growth Hacker Tools
  129. How To Make Hacking Tools
  130. Hacking Tools Free Download
  131. Bluetooth Hacking Tools Kali
  132. Bluetooth Hacking Tools Kali
  133. Hackrf Tools
  134. Hacker Security Tools
  135. Hacking Tools Kit
  136. Pentest Tools For Ubuntu
  137. Pentest Tools Alternative
  138. Hacker Tools Online
  139. Hack Tools Mac
  140. Hacker Tools Software
  141. Hack Tools
  142. Best Hacking Tools 2019
  143. Hacker Tools Apk
  144. Top Pentest Tools
  145. Pentest Tools Website
  146. Pentest Tools Kali Linux
  147. Hacker
  148. Install Pentest Tools Ubuntu
  149. Hack Tools For Ubuntu
  150. Pentest Tools Android
  151. Hacker Tools Hardware
  152. Hacking Tools For Pc
  153. Hacking Tools Download
  154. Black Hat Hacker Tools
  155. Github Hacking Tools
  156. Install Pentest Tools Ubuntu
  157. Pentest Tools Apk
  158. Nsa Hacker Tools
  159. Hack And Tools
  160. Pentest Tools Kali Linux
  161. Hacker Tool Kit
  162. Hacking Tools Mac
  163. Nsa Hack Tools Download
  164. Hack Tools
  165. Hacker Tools Mac
  166. Easy Hack Tools
  167. Hack Tools
  168. Hacker Tools Mac
  169. Hacking Tools Online
  170. How To Install Pentest Tools In Ubuntu
  171. Hacker Tools Hardware
  172. Hacker Tools Hardware
  173. Hacks And Tools
  174. Hacker Tools Apk Download
  175. Hacking Tools Online

0 comments:

Post a Comment

Subscribe to: Post Comments (Atom)