Gitiles
Code Review Sign In
source.supwisdom.com / UniIdentify / tomcat / 6266f996613ae5631b7288523f895b36e30fe5af / . / tomcat-uid / webapps / docs / cluster-howto.html
blob: ac90474bea66d6c46e3e82b487b42247ed49f9e0 [file] [log] [blame]
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]1<html><head><META http-equiv="Content-Type" content="text/html; charset=iso-8859-1"><title>Apache Tomcat 7 (7.0.77) - Clustering/Session Replication HOW-TO</title><meta name="author" content="Filip Hanik"><meta name="author" content="Peter Rossbach"><style type="text/css" media="print">
2 .noPrint {display: none;}
3 td#mainBody {width: 100%;}
4</style><style type="text/css">
5code {background-color:rgb(224,255,255);padding:0 0.1em;}
6code.attributeName, code.propertyName {background-color:transparent;}
7
8
9table {
10 border-collapse: collapse;
11 text-align: left;
12}
13table *:not(table) {
14 /* Prevent border-collapsing for table child elements like <div> */
15 border-collapse: separate;
16}
17
18th {
19 text-align: left;
20}
21
22
23div.codeBox pre code, code.attributeName, code.propertyName, code.noHighlight, .noHighlight code {
24 background-color: transparent;
25}
26div.codeBox {
27 overflow: auto;
28 margin: 1em 0;
29}
30div.codeBox pre {
31 margin: 0;
32 padding: 4px;
33 border: 1px solid #999;
34 border-radius: 5px;
35 background-color: #eff8ff;
36 display: table; /* To prevent <pre>s from taking the complete available width. */
37 /*
38 When it is officially supported, use the following CSS instead of display: table
39 to prevent big <pre>s from exceeding the browser window:
40 max-width: available;
41 width: min-content;
42 */
43}
44
45div.codeBox pre.wrap {
46 white-space: pre-wrap;
47}
48
49
50table.defaultTable tr, table.detail-table tr {
51 border: 1px solid #CCC;
52}
53
54table.defaultTable tr:nth-child(even), table.detail-table tr:nth-child(even) {
55 background-color: #FAFBFF;
56}
57
58table.defaultTable tr:nth-child(odd), table.detail-table tr:nth-child(odd) {
59 background-color: #EEEFFF;
60}
61
62table.defaultTable th, table.detail-table th {
63 background-color: #88b;
64 color: #fff;
65}
66
67table.defaultTable th, table.defaultTable td, table.detail-table th, table.detail-table td {
68 padding: 5px 8px;
69}
70
71
72p.notice {
73 border: 1px solid rgb(255, 0, 0);
74 background-color: rgb(238, 238, 238);
75 color: rgb(0, 51, 102);
76 padding: 0.5em;
77 margin: 1em 2em 1em 1em;
78}
79</style></head><body bgcolor="#ffffff" text="#000000" link="#525D76" alink="#525D76" vlink="#525D76"><table border="0" width="100%" cellspacing="0"><!--PAGE HEADER--><tr><td><!--PROJECT LOGO--><a href="http://tomcat.apache.org/"><img src="./images/tomcat.gif" align="right" alt="
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]80 The Apache Tomcat Servlet/JSP Container
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]81 " border="0"></a></td><td><h1><font face="arial,helvetica,sanserif">Apache Tomcat 7</font></h1><font face="arial,helvetica,sanserif">Version 7.0.77, Mar 28 2017</font></td><td><!--APACHE LOGO--><a href="http://www.apache.org/"><img src="./images/asf-logo.svg" align="right" alt="Apache Logo" border="0" style="width: 266px;height: 83px;"></a></td></tr></table><table border="0" width="100%" cellspacing="4"><!--HEADER SEPARATOR--><tr><td colspan="2"><hr noshade size="1"></td></tr><tr><!--LEFT SIDE NAVIGATION--><td width="20%" valign="top" nowrap class="noPrint"><p><strong>Links</strong></p><ul><li><a href="index.html">Docs Home</a></li><li><a href="http://wiki.apache.org/tomcat/FAQ">FAQ</a></li><li><a href="#comments_section">User Comments</a></li></ul><p><strong>User Guide</strong></p><ul><li><a href="introduction.html">1) Introduction</a></li><li><a href="setup.html">2) Setup</a></li><li><a href="appdev/index.html">3) First webapp</a></li><li><a href="deployer-howto.html">4) Deployer</a></li><li><a href="manager-howto.html">5) Manager</a></li><li><a href="realm-howto.html">6) Realms and AAA</a></li><li><a href="security-manager-howto.html">7) Security Manager</a></li><li><a href="jndi-resources-howto.html">8) JNDI Resources</a></li><li><a href="jndi-datasource-examples-howto.html">9) JDBC DataSources</a></li><li><a href="class-loader-howto.html">10) Classloading</a></li><li><a href="jasper-howto.html">11) JSPs</a></li><li><a href="ssl-howto.html">12) SSL/TLS</a></li><li><a href="ssi-howto.html">13) SSI</a></li><li><a href="cgi-howto.html">14) CGI</a></li><li><a href="proxy-howto.html">15) Proxy Support</a></li><li><a href="mbeans-descriptors-howto.html">16) MBeans Descriptors</a></li><li><a href="default-servlet.html">17) Default Servlet</a></li><li><a href="cluster-howto.html">18) Clustering</a></li><li><a href="balancer-howto.html">19) Load Balancer</a></li><li><a href="connectors.html">20) Connectors</a></li><li><a href="monitoring.html">21) Monitoring and Management</a></li><li><a href="logging.html">22) Logging</a></li><li><a href="apr.html">23) APR/Native</a></li><li><a href="virtual-hosting-howto.html">24) Virtual Hosting</a></li><li><a href="aio.html">25) Advanced IO</a></li><li><a href="extras.html">26) Additional Components</a></li><li><a href="maven-jars.html">27) Mavenized</a></li><li><a href="security-howto.html">28) Security Considerations</a></li><li><a href="windows-service-howto.html">29) Windows Service</a></li><li><a href="windows-auth-howto.html">30) Windows Authentication</a></li><li><a href="jdbc-pool.html">31) Tomcat's JDBC Pool</a></li><li><a href="web-socket-howto.html">32) WebSocket</a></li></ul><p><strong>Reference</strong></p><ul><li><a href="RELEASE-NOTES.txt">Release Notes</a></li><li><a href="config/index.html">Configuration</a></li><li><a href="api/index.html">Tomcat Javadocs</a></li><li><a href="servletapi/index.html">Servlet Javadocs</a></li><li><a href="jspapi/index.html">JSP 2.2 Javadocs</a></li><li><a href="elapi/index.html">EL 2.2 Javadocs</a></li><li><a href="websocketapi/index.html">WebSocket 1.1 Javadocs</a></li><li><a href="http://tomcat.apache.org/connectors-doc/">JK 1.2 Documentation</a></li></ul><p><strong>Apache Tomcat Development</strong></p><ul><li><a href="building.html">Building</a></li><li><a href="changelog.html">Changelog</a></li><li><a href="http://wiki.apache.org/tomcat/TomcatVersions">Status</a></li><li><a href="developers.html">Developers</a></li><li><a href="architecture/index.html">Architecture</a></li><li><a href="funcspecs/index.html">Functional Specs.</a></li><li><a href="tribes/introduction.html">Tribes</a></li></ul></td><!--RIGHT SIDE MAIN BODY--><td width="80%" valign="top" align="left" id="mainBody"><h1>Clustering/Session Replication HOW-TO</h1><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="Important Note"><!--()--></a><a name="Important_Note"><strong>Important Note</strong></a></font></td></tr><tr><td><blockquote>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]82<p><b>You can also check the <a href="config/cluster.html">configuration reference documentation.</a></b>
83</p>
84</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="Table of Contents"><!--()--></a><a name="Table_of_Contents"><strong>Table of Contents</strong></a></font></td></tr><tr><td><blockquote>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]85<ul><li><a href="#For_the_impatient">For the impatient</a></li><li><a href="#Security">Security</a></li><li><a href="#Cluster_Basics">Cluster Basics</a></li><li><a href="#Overview">Overview</a></li><li><a href="#Cluster_Information">Cluster Information</a></li><li><a href="#Bind_session_after_crash_to_failover_node">Bind session after crash to failover node</a></li><li><a href="#Configuration_Example">Configuration Example</a></li><li><a href="#Cluster_Architecture">Cluster Architecture</a></li><li><a href="#How_it_Works">How it Works</a></li><li><a href="#Monitoring_your_Cluster_with_JMX">Monitoring your Cluster with JMX</a></li><li><a href="#FAQ">FAQ</a></li></ul>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]86</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="For the impatient"><!--()--></a><a name="For_the_impatient"><strong>For the impatient</strong></a></font></td></tr><tr><td><blockquote>
87 <p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]88 Simply add
89 </p>
90 <div class="codeBox"><pre><code>&lt;Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/&gt;</code></pre></div>
91 <p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]92 to your <code>&lt;Engine&gt;</code> or your <code>&lt;Host&gt;</code> element to enable clustering.
93 </p>
94 <p>
95 Using the above configuration will enable all-to-all session replication
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]96 using the <code>DeltaManager</code> to replicate session deltas. By all-to-all we mean that the session gets replicated to all the other
97 nodes in the cluster. This works great for smaller cluster but we don't recommend it for larger clusters(a lot of Tomcat nodes).
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]98 Also when using the delta manager it will replicate to all nodes, even nodes that don't have the application deployed.<br>
99 To get around this problem, you'll want to use the BackupManager. This manager only replicates the session data to one backup
100 node, and only to nodes that have the application deployed. Downside of the BackupManager: not quite as battle tested as the delta manager.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]101 </p>
102 <p>
103 Here are some of the important default values:
104 </p>
105 <ol>
106 <li>Multicast address is 228.0.0.4</li>
107 <li>Multicast port is 45564 (the port and the address together determine cluster membership.</li>
108 <li>The IP broadcasted is <code>java.net.InetAddress.getLocalHost().getHostAddress()</code> (make sure you don't broadcast 127.0.0.1, this is a common error)</li>
109 <li>The TCP port listening for replication messages is the first available server socket in range <code>4000-4100</code></li>
110 <li>Listener is configured <code>ClusterSessionListener</code></li>
111 <li>Two interceptors are configured <code>TcpFailureDetector</code> and <code>MessageDispatch15Interceptor</code></li>
112 </ol>
113 <p>
114 The following is the default cluster configuration:
115 </p>
116 <div class="codeBox"><pre><code> &lt;Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]117 channelSendOptions="8"&gt;
118
119 &lt;Manager className="org.apache.catalina.ha.session.DeltaManager"
120 expireSessionsOnShutdown="false"
121 notifyListenersOnReplication="true"/&gt;
122
123 &lt;Channel className="org.apache.catalina.tribes.group.GroupChannel"&gt;
124 &lt;Membership className="org.apache.catalina.tribes.membership.McastService"
125 address="228.0.0.4"
126 port="45564"
127 frequency="500"
128 dropTime="3000"/&gt;
129 &lt;Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
130 address="auto"
131 port="4000"
132 autoBind="100"
133 selectorTimeout="5000"
134 maxThreads="6"/&gt;
135
136 &lt;Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter"&gt;
137 &lt;Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/&gt;
138 &lt;/Sender&gt;
139 &lt;Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/&gt;
140 &lt;Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/&gt;
141 &lt;/Channel&gt;
142
143 &lt;Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
144 filter=""/&gt;
145 &lt;Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/&gt;
146
147 &lt;Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
148 tempDir="/tmp/war-temp/"
149 deployDir="/tmp/war-deploy/"
150 watchDir="/tmp/war-listen/"
151 watchEnabled="false"/&gt;
152
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]153 &lt;ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"&gt;
154 &lt;ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"&gt;
155 &lt;/Cluster&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]156 <p>Will cover this section in more detail later in this document.</p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]157</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="Security"><strong>Security</strong></a></font></td></tr><tr><td><blockquote>
158
159<p>The cluster implementation is written on the basis that a secure, trusted
160network is used for all of the cluster related network traffic. It is not safe
161to run a cluster on a insecure, untrusted network.</p>
162
163<p>There are many options for providing a secure, trusted network for use by a
164Tomcat cluster. These include:</p>
165<ul>
166 <li>private LAN</li>
167 <li>a Virtual Private Network (VPN)</li>
168 <li>IPSEC</li>
169</ul>
170
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]171</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="Cluster Basics"><!--()--></a><a name="Cluster_Basics"><strong>Cluster Basics</strong></a></font></td></tr><tr><td><blockquote>
172
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]173<p>To run session replication in your Tomcat 7.0 container, the following steps
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]174should be completed:</p>
175<ul>
176 <li>All your session attributes must implement <code>java.io.Serializable</code></li>
177 <li>Uncomment the <code>Cluster</code> element in server.xml</li>
178 <li>If you have defined custom cluster valves, make sure you have the <code>ReplicationValve</code> defined as well under the Cluster element in server.xml</li>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]179 <li>If your Tomcat instances are running on the same machine, make sure the <code>Receiver.port</code>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]180 attribute is unique for each instance, in most cases Tomcat is smart enough to resolve this on it's own by autodetecting available ports in the range 4000-4100</li>
181 <li>Make sure your <code>web.xml</code> has the
182 <code>&lt;distributable/&gt;</code> element</li>
183 <li>If you are using mod_jk, make sure that jvmRoute attribute is set at your Engine <code>&lt;Engine name="Catalina" jvmRoute="node01" &gt;</code>
184 and that the jvmRoute attribute value matches your worker name in workers.properties</li>
185 <li>Make sure that all nodes have the same time and sync with NTP service!</li>
186 <li>Make sure that your loadbalancer is configured for sticky session mode.</li>
187</ul>
188<p>Load balancing can be achieved through many techniques, as seen in the
189<a href="balancer-howto.html">Load Balancing</a> chapter.</p>
190<p>Note: Remember that your session state is tracked by a cookie, so your URL must look the same from the out
191 side otherwise, a new session will be created.</p>
192<p>Note: Clustering support currently requires the JDK version 1.5 or later.</p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]193<p>The Cluster module uses the Tomcat JULI logging framework, so you can configure logging
194 through the regular logging.properties file. To track messages, you can enable logging on the key: <code>org.apache.catalina.tribes.MESSAGES</code></p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]195</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="Overview"><strong>Overview</strong></a></font></td></tr><tr><td><blockquote>
196
197<p>To enable session replication in Tomcat, three different paths can be followed to achieve the exact same thing:</p>
198<ol>
199 <li>Using session persistence, and saving the session to a shared file system (PersistenceManager + FileStore)</li>
200 <li>Using session persistence, and saving the session to a shared database (PersistenceManager + JDBCStore)</li>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]201 <li>Using in-memory-replication, using the SimpleTcpCluster that ships with Tomcat (lib/catalina-tribes.jar + lib/catalina-ha.jar)</li>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]202</ol>
203
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]204<p>In this release of session replication, Tomcat can perform an all-to-all replication of session state using the <code>DeltaManager</code> or
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]205 perform backup replication to only one node using the <code>BackupManager</code>.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]206 The all-to-all replication is an algorithm that is only efficient when the clusters are small. For larger clusters, to use
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]207 a primary-secondary session replication where the session will only be stored at one backup server simply setup the BackupManager. <br>
208 Currently you can use the domain worker attribute (mod_jk &gt; 1.2.8) to build cluster partitions
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]209 with the potential of having a more scalable cluster solution with the DeltaManager(you'll need to configure the domain interceptor for this).
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]210 In order to keep the network traffic down in an all-to-all environment, you can split your cluster
211 into smaller groups. This can be easily achieved by using different multicast addresses for the different groups.
212 A very simple setup would look like this:
213 </p>
214
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]215<div class="codeBox"><pre><code> DNS Round Robin
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]216 |
217 Load Balancer
218 / \
219 Cluster1 Cluster2
220 / \ / \
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]221 Tomcat1 Tomcat2 Tomcat3 Tomcat4</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]222
223<p>What is important to mention here, is that session replication is only the beginning of clustering.
224 Another popular concept used to implement clusters is farming, i.e., you deploy your apps only to one
225 server, and the cluster will distribute the deployments across the entire cluster.
226 This is all capabilities that can go into with the FarmWarDeployer (s. cluster example at <code>server.xml</code>)</p>
227<p>In the next section will go deeper into how session replication works and how to configure it.</p>
228
229</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="Cluster Information"><!--()--></a><a name="Cluster_Information"><strong>Cluster Information</strong></a></font></td></tr><tr><td><blockquote>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]230<p>Membership is established using multicast heartbeats.
231 Hence, if you wish to subdivide your clusters, you can do this by
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]232 changing the multicast IP address or port in the <code>&lt;Membership&gt;</code> element.
233</p>
234<p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]235 The heartbeat contains the IP address of the Tomcat node and the TCP port that
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]236 Tomcat listens to for replication traffic. All data communication happens over TCP.
237</p>
238<p>
239 The <code>ReplicationValve</code> is used to find out when the request has been completed and initiate the
240 replication, if any. Data is only replicated if the session has changed (by calling setAttribute or removeAttribute
241 on the session).
242</p>
243<p>
244 One of the most important performance considerations is the synchronous versus asynchronous replication.
245 In a synchronous replication mode the request doesn't return until the replicated session has been
246 sent over the wire and reinstantiated on all the other cluster nodes.
247 Synchronous vs. asynchronous is configured using the <code>channelSendOptions</code>
248 flag and is an integer value. The default value for the <code>SimpleTcpCluster/DeltaManager</code> combo is
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]249 8, which is asynchronous. You can read more on the <a href="tribes/introduction.html">send flag(overview)</a> or the
250 <a href="http://tomcat.apache.org/tomcat-7.0-doc/api/org/apache/catalina/tribes/Channel.html">send flag(javadoc)</a>.
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]251 During async replication, the request is returned before the data has been replicated. async replication yields shorter
252 request times, and synchronous replication guarantees the session to be replicated before the request returns.
253</p>
254</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="Bind session after crash to failover node"><!--()--></a><a name="Bind_session_after_crash_to_failover_node"><strong>Bind session after crash to failover node</strong></a></font></td></tr><tr><td><blockquote>
255<p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]256 If you are using mod_jk and not using sticky sessions or for some reasons sticky session don't
257 work, or you are simply failing over, the session id will need to be modified as it previously contained
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]258 the worker id of the previous tomcat (as defined by jvmRoute in the Engine element).
259 To solve this, we will use the JvmRouteBinderValve.
260</p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]261<p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]262 The JvmRouteBinderValve rewrites the session id to ensure that the next request will remain sticky
263 (and not fall back to go to random nodes since the worker is no longer available) after a fail over.
264 The valve rewrites the JSESSIONID value in the cookie with the same name.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]265 Not having this valve in place, will make it harder to ensure stickiness in case of a failure for the mod_jk module.
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]266</p>
267<p>
268 By default, if no valves are configured, the JvmRouteBinderValve is added on.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]269 The cluster message listener called JvmRouteSessionIDBinderListener is also defined by default and is used to actually rewrite the
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]270 session id on the other nodes in the cluster once a fail over has occurred.
271 Remember, if you are adding your own valves or cluster listeners in server.xml then the defaults are no longer valid,
272 make sure that you add in all the appropriate valves and listeners as defined by the default.
273</p>
274<p>
275 <b>Hint:</b><br>
276 With attribute <i>sessionIdAttribute</i> you can change the request attribute name that included the old session id.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]277 Default attribute name is <i>org.apache.catalina.ha.session.JvmRouteOrignalSessionID</i>.
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]278</p>
279<p>
280 <b>Trick:</b><br>
281 You can enable this mod_jk turnover mode via JMX before you drop a node to all backup nodes!
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]282 Set enable true on all JvmRouteBinderValve backups, disable worker at mod_jk
283 and then drop node and restart it! Then enable mod_jk Worker and disable JvmRouteBinderValves again.
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]284 This use case means that only requested session are migrated.
285</p>
286
287
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]288</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="Configuration Example"><!--()--></a><a name="Configuration_Example"><strong>Configuration Example</strong></a></font></td></tr><tr><td><blockquote>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]289 <div class="codeBox"><pre><code> &lt;Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]290 channelSendOptions="6"&gt;
291
292 &lt;Manager className="org.apache.catalina.ha.session.BackupManager"
293 expireSessionsOnShutdown="false"
294 notifyListenersOnReplication="true"
295 mapSendOptions="6"/&gt;
296 &lt;!--
297 &lt;Manager className="org.apache.catalina.ha.session.DeltaManager"
298 expireSessionsOnShutdown="false"
299 notifyListenersOnReplication="true"/&gt;
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]300 --&gt;
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]301 &lt;Channel className="org.apache.catalina.tribes.group.GroupChannel"&gt;
302 &lt;Membership className="org.apache.catalina.tribes.membership.McastService"
303 address="228.0.0.4"
304 port="45564"
305 frequency="500"
306 dropTime="3000"/&gt;
307 &lt;Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
308 address="auto"
309 port="5000"
310 selectorTimeout="100"
311 maxThreads="6"/&gt;
312
313 &lt;Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter"&gt;
314 &lt;Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/&gt;
315 &lt;/Sender&gt;
316 &lt;Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/&gt;
317 &lt;Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/&gt;
318 &lt;Interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/&gt;
319 &lt;/Channel&gt;
320
321 &lt;Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]322 filter=".*\.gif|.*\.js|.*\.jpeg|.*\.jpg|.*\.png|.*\.htm|.*\.html|.*\.css|.*\.txt"/&gt;
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]323
324 &lt;Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
325 tempDir="/tmp/war-temp/"
326 deployDir="/tmp/war-deploy/"
327 watchDir="/tmp/war-listen/"
328 watchEnabled="false"/&gt;
329
330 &lt;ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/&gt;
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]331 &lt;/Cluster&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]332 <p>
333 Break it down!!
334 </p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]335 <div class="codeBox"><pre><code> &lt;Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
336 channelSendOptions="6"&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]337 <p>
338 The main element, inside this element all cluster details can be configured.
339 The <code>channelSendOptions</code> is the flag that is attached to each message sent by the
340 SimpleTcpCluster class or any objects that are invoking the SimpleTcpCluster.send method.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]341 The description of the send flags is available at <a href="http://tomcat.apache.org/tomcat-7.0-doc/api/org/apache/catalina/tribes/Channel.html">
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]342 our javadoc site</a>
343 The <code>DeltaManager</code> sends information using the SimpleTcpCluster.send method, while the backup manager
344 sends it itself directly through the channel.
345 <br>For more info, Please visit the <a href="config/cluster.html">reference documentation</a>
346 </p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]347 <div class="codeBox"><pre><code> &lt;Manager className="org.apache.catalina.ha.session.BackupManager"
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]348 expireSessionsOnShutdown="false"
349 notifyListenersOnReplication="true"
350 mapSendOptions="6"/&gt;
351 &lt;!--
352 &lt;Manager className="org.apache.catalina.ha.session.DeltaManager"
353 expireSessionsOnShutdown="false"
354 notifyListenersOnReplication="true"/&gt;
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]355 --&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]356 <p>
357 This is a template for the manager configuration that will be used if no manager is defined in the &lt;Context&gt;
358 element. In Tomcat 5.x each webapp marked distributable had to use the same manager, this is no longer the case
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]359 since Tomcat you can define a manager class for each webapp, so that you can mix managers in your cluster.
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]360 Obviously the managers on one node's application has to correspond with the same manager on the same application on the other node.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]361 If no manager has been specified for the webapp, and the webapp is marked &lt;distributable/&gt; Tomcat will take this manager configuration
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]362 and create a manager instance cloning this configuration.
363 <br>For more info, Please visit the <a href="config/cluster-manager.html">reference documentation</a>
364 </p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]365 <div class="codeBox"><pre><code> &lt;Channel className="org.apache.catalina.tribes.group.GroupChannel"&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]366 <p>
367 The channel element is <a href="tribes/introduction.html">Tribes</a>, the group communication framework
368 used inside Tomcat. This element encapsulates everything that has to do with communication and membership logic.
369 <br>For more info, Please visit the <a href="config/cluster-channel.html">reference documentation</a>
370 </p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]371 <div class="codeBox"><pre><code> &lt;Membership className="org.apache.catalina.tribes.membership.McastService"
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]372 address="228.0.0.4"
373 port="45564"
374 frequency="500"
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]375 dropTime="3000"/&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]376 <p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]377 Membership is done using multicasting. Please note that Tribes also supports static memberships using the
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]378 <code>StaticMembershipInterceptor</code> if you want to extend your membership to points beyond multicasting.
379 The address attribute is the multicast address used and the port is the multicast port. These two together
380 create the cluster separation. If you want a QA cluster and a production cluster, the easiest config is to
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]381 have the QA cluster be on a separate multicast address/port combination than the production cluster.<br>
382 The membership component broadcasts TCP address/port of itself to the other nodes so that communication between
383 nodes can be done over TCP. Please note that the address being broadcasted is the one of the
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]384 <code>Receiver.address</code> attribute.
385 <br>For more info, Please visit the <a href="config/cluster-membership.html">reference documentation</a>
386 </p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]387 <div class="codeBox"><pre><code> &lt;Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]388 address="auto"
389 port="5000"
390 selectorTimeout="100"
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]391 maxThreads="6"/&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]392 <p>
393 In tribes the logic of sending and receiving data has been broken into two functional components. The Receiver, as the name suggests
394 is responsible for receiving messages. Since the Tribes stack is thread less, (a popular improvement now adopted by other frameworks as well),
395 there is a thread pool in this component that has a maxThreads and minThreads setting.<br>
396 The address attribute is the host address that will be broadcasted by the membership component to the other nodes.
397 <br>For more info, Please visit the <a href="config/cluster-receiver.html">reference documentation</a>
398 </p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]399 <div class="codeBox"><pre><code> &lt;Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter"&gt;
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]400 &lt;Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/&gt;
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]401 &lt;/Sender&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]402 <p>
403 The sender component, as the name indicates is responsible for sending messages to other nodes.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]404 The sender has a shell component, the <code>ReplicationTransmitter</code> but the real stuff done is done in the
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]405 sub component, <code>Transport</code>.
406 Tribes support having a pool of senders, so that messages can be sent in parallel and if using the NIO sender,
407 you can send messages concurrently as well.<br>
408 Concurrently means one message to multiple senders at the same time and Parallel means multiple messages to multiple senders
409 at the same time.
410 <br>For more info, Please visit the <a href="config/cluster-sender.html">reference documentation</a>
411 </p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]412 <div class="codeBox"><pre><code> &lt;Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/&gt;
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]413 &lt;Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/&gt;
414 &lt;Interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/&gt;
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]415 &lt;/Channel&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]416 <p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]417 Tribes uses a stack to send messages through. Each element in the stack is called an interceptor, and works much like the valves do
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]418 in the Tomcat servlet container.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]419 Using interceptors, logic can be broken into more manageable pieces of code. The interceptors configured above are:<br>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]420 TcpFailureDetector - verifies crashed members through TCP, if multicast packets get dropped, this interceptor protects against false positives,
421 ie the node marked as crashed even though it still is alive and running.<br>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]422 MessageDispatch15Interceptor - dispatches messages to a thread (thread pool) to send message asynchronously.<br>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]423 ThroughputInterceptor - prints out simple stats on message traffic.<br>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]424 Please note that the order of interceptors is important. The way they are defined in server.xml is the way they are represented in the
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]425 channel stack. Think of it as a linked list, with the head being the first most interceptor and the tail the last.
426 <br>For more info, Please visit the <a href="config/cluster-interceptor.html">reference documentation</a>
427 </p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]428 <div class="codeBox"><pre><code> &lt;Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
429 filter=".*\.gif|.*\.js|.*\.jpeg|.*\.jpg|.*\.png|.*\.htm|.*\.html|.*\.css|.*\.txt"/&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]430 <p>
431 The cluster uses valves to track requests to web applications, we've mentioned the ReplicationValve and the JvmRouteBinderValve above.
432 The &lt;Cluster&gt; element itself is not part of the pipeline in Tomcat, instead the cluster adds the valve to its parent container.
433 If the &lt;Cluster&gt; elements is configured in the &lt;Engine&gt; element, the valves get added to the engine and so on.
434 <br>For more info, Please visit the <a href="config/cluster-valve.html">reference documentation</a>
435 </p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]436 <div class="codeBox"><pre><code> &lt;Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]437 tempDir="/tmp/war-temp/"
438 deployDir="/tmp/war-deploy/"
439 watchDir="/tmp/war-listen/"
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]440 watchEnabled="false"/&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]441 <p>
442 The default tomcat cluster supports farmed deployment, ie, the cluster can deploy and undeploy applications on the other nodes.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]443 The state of this component is currently in flux but will be addressed soon. There was a change in the deployment algorithm
444 between Tomcat 5.0 and 5.5 and at that point, the logic of this component changed to where the deploy dir has to match the
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]445 webapps directory.
446 <br>For more info, Please visit the <a href="config/cluster-deployer.html">reference documentation</a>
447 </p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]448 <div class="codeBox"><pre><code> &lt;ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/&gt;
449 &lt;/Cluster&gt;</code></pre></div>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]450 <p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]451 Since the SimpleTcpCluster itself is a sender and receiver of the Channel object, components can register themselves as listeners to
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]452 the SimpleTcpCluster. The listener above <code>ClusterSessionListener</code> listens for DeltaManager replication messages
453 and applies the deltas to the manager that in turn applies it to the session.
454 <br>For more info, Please visit the <a href="config/cluster-listener.html">reference documentation</a>
455 </p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]456
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]457</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="Cluster Architecture"><!--()--></a><a name="Cluster_Architecture"><strong>Cluster Architecture</strong></a></font></td></tr><tr><td><blockquote>
458
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]459<p><b>Component Levels:</b></p>
460<div class="codeBox"><pre><code> Server
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]461 |
462 Service
463 |
464 Engine
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]465 | \
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]466 | --- Cluster --*
467 |
468 Host
469 |
470 ------
471 / \
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]472 Cluster Context(1-N)
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]473 | \
474 | -- Manager
475 | \
476 | -- DeltaManager
477 | -- BackupManager
478 |
479 ---------------------------
480 | \
481 Channel \
482 ----------------------------- \
483 | \
484 Interceptor_1 .. \
485 | \
486 Interceptor_N \
487 ----------------------------- \
488 | | | \
489 Receiver Sender Membership \
490 -- Valve
491 | \
492 | -- ReplicationValve
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]493 | -- JvmRouteBinderValve
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]494 |
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]495 -- LifecycleListener
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]496 |
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]497 -- ClusterListener
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]498 | \
499 | -- ClusterSessionListener
500 | -- JvmRouteSessionIDBinderListener
501 |
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]502 -- Deployer
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]503 \
504 -- FarmWarDeployer
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]505
506</code></pre></div>
507
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]508
509</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="How it Works"><!--()--></a><a name="How_it_Works"><strong>How it Works</strong></a></font></td></tr><tr><td><blockquote>
510<p>To make it easy to understand how clustering works, We are gonna take you through a series of scenarios.
511 In the scenario we only plan to use two tomcat instances <code>TomcatA</code> and <code>TomcatB</code>.
512 We will cover the following sequence of events:</p>
513
514<ol>
515<li><code>TomcatA</code> starts up</li>
516<li><code>TomcatB</code> starts up (Wait that TomcatA start is complete)</li>
517<li><code>TomcatA</code> receives a request, a session <code>S1</code> is created.</li>
518<li><code>TomcatA</code> crashes</li>
519<li><code>TomcatB</code> receives a request for session <code>S1</code></li>
520<li><code>TomcatA</code> starts up</li>
521<li><code>TomcatA</code> receives a request, invalidate is called on the session (<code>S1</code>)</li>
522<li><code>TomcatB</code> receives a request, for a new session (<code>S2</code>)</li>
523<li><code>TomcatA</code> The session <code>S2</code> expires due to inactivity.</li>
524</ol>
525
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]526<p>Ok, now that we have a good sequence, we will take you through exactly what happens in the session replication code</p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]527
528<ol>
529<li><b><code>TomcatA</code> starts up</b>
530 <p>
531 Tomcat starts up using the standard start up sequence. When the Host object is created, a cluster object is associated with it.
532 When the contexts are parsed, if the distributable element is in place in web.xml
533 Tomcat asks the Cluster class (in this case <code>SimpleTcpCluster</code>) to create a manager
534 for the replicated context. So with clustering enabled, distributable set in web.xml
535 Tomcat will create a <code>DeltaManager</code> for that context instead of a <code>StandardManager</code>.
536 The cluster class will start up a membership service (multicast) and a replication service (tcp unicast).
537 More on the architecture further down in this document.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]538 </p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]539</li>
540<li><b><code>TomcatB</code> starts up</b>
541 <p>
542 When TomcatB starts up, it follows the same sequence as TomcatA did with one exception.
543 The cluster is started and will establish a membership (TomcatA,TomcatB).
544 TomcatB will now request the session state from a server that already exists in the cluster,
545 in this case TomcatA. TomcatA responds to the request, and before TomcatB starts listening
546 for HTTP requests, the state has been transferred from TomcatA to TomcatB.
547 In case TomcatA doesn't respond, TomcatB will time out after 60 seconds, and issue a log
548 entry. The session state gets transferred for each web application that has distributable in
549 its web.xml. Note: To use session replication efficiently, all your tomcat instances should be
550 configured the same.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]551 </p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]552</li>
553<li><B><code>TomcatA</code> receives a request, a session <code>S1</code> is created.</B>
554 <p>
555 The request coming in to TomcatA is treated exactly the same way as without session replication.
556 The action happens when the request is completed, the <code>ReplicationValve</code> will intercept
557 the request before the response is returned to the user.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]558 At this point it finds that the session has been modified, and it uses TCP to replicate the
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]559 session to TomcatB. Once the serialized data has been handed off to the operating systems TCP logic,
560 the request returns to the user, back through the valve pipeline.
561 For each request the entire session is replicated, this allows code that modifies attributes
562 in the session without calling setAttribute or removeAttribute to be replicated.
563 a useDirtyFlag configuration parameter can be used to optimize the number of times
564 a session is replicated.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]565 </p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]566
567</li>
568<li><b><code>TomcatA</code> crashes</b>
569 <p>
570 When TomcatA crashes, TomcatB receives a notification that TomcatA has dropped out
571 of the cluster. TomcatB removes TomcatA from its membership list, and TomcatA will no longer
572 be notified of any changes that occurs in TomcatB.
573 The load balancer will redirect the requests from TomcatA to TomcatB and all the sessions
574 are current.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]575 </p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]576</li>
577<li><b><code>TomcatB</code> receives a request for session <code>S1</code></b>
578 <p>Nothing exciting, TomcatB will process the request as any other request.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]579 </p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]580</li>
581<li><b><code>TomcatA</code> starts up</b>
582 <p>Upon start up, before TomcatA starts taking new request and making itself
583 available to it will follow the start up sequence described above 1) 2).
584 It will join the cluster, contact TomcatB for the current state of all the sessions.
585 And once it receives the session state, it finishes loading and opens its HTTP/mod_jk ports.
586 So no requests will make it to TomcatA until it has received the session state from TomcatB.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]587 </p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]588</li>
589<li><b><code>TomcatA</code> receives a request, invalidate is called on the session (<code>S1</code>)</b>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]590 <p>The invalidate call is intercepted, and the session is queued with invalidated sessions.
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]591 When the request is complete, instead of sending out the session that has changed, it sends out
592 an "expire" message to TomcatB and TomcatB will invalidate the session as well.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]593 </p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]594
595</li>
596<li><b><code>TomcatB</code> receives a request, for a new session (<code>S2</code>)</b>
597 <p>Same scenario as in step 3)
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]598 </p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]599
600
601</li>
602<li><code>TomcatA</code> The session <code>S2</code> expires due to inactivity.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]603 <p>The invalidate call is intercepted the same was as when a session is invalidated by the user,
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]604 and the session is queued with invalidated sessions.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]605 At this point, the invalidated session will not be replicated across until
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]606 another request comes through the system and checks the invalid queue.
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]607 </p>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]608</li>
609</ol>
610
611<p>Phuuuhh! :)</p>
612
613<p><b>Membership</b>
614 Clustering membership is established using very simple multicast pings.
615 Each Tomcat instance will periodically send out a multicast ping,
616 in the ping message the instance will broad cast its IP and TCP listen port
617 for replication.
618 If an instance has not received such a ping within a given timeframe, the
619 member is considered dead. Very simple, and very effective!
620 Of course, you need to enable multicasting on your system.
621</p>
622
623<p><b>TCP Replication</b>
624 Once a multicast ping has been received, the member is added to the cluster
625 Upon the next replication request, the sending instance will use the host and
626 port info and establish a TCP socket. Using this socket it sends over the serialized data.
627 The reason I choose TCP sockets is because it has built in flow control and guaranteed delivery.
628 So I know, when I send some data, it will make it there :)
629</p>
630
631<p><b>Distributed locking and pages using frames</b>
632 Tomcat does not keep session instances in sync across the cluster.
633 The implementation of such logic would be to much overhead and cause all
634 kinds of problems. If your client accesses the same session
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]635 simultaneously using multiple requests, then the last request
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]636 will override the other sessions in the cluster.
637</p>
638
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]639</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="Monitoring your Cluster with JMX"><!--()--></a><a name="Monitoring_your_Cluster_with_JMX"><strong>Monitoring your Cluster with JMX</strong></a></font></td></tr><tr><td><blockquote>
640<p>Monitoring is a very important question when you use a cluster. Some of the cluster objects are JMX MBeans </p>
641<p>Add the following parameter to your startup script with Java 5:</p>
642<div class="codeBox"><pre><code>set CATALINA_OPTS=\
643-Dcom.sun.management.jmxremote \
644-Dcom.sun.management.jmxremote.port=%my.jmx.port% \
645-Dcom.sun.management.jmxremote.ssl=false \
646-Dcom.sun.management.jmxremote.authenticate=false</code></pre></div>
647
648<p>
649 List of Cluster Mbeans
650</p>
651<table class="defaultTable">
652
653 <tr>
654 <th>Name</th>
655 <th>Description</th>
656 <th>MBean ObjectName - Engine</th>
657 <th>MBean ObjectName - Host</th>
658 </tr>
659
660 <tr>
661 <td>Cluster</td>
662 <td>The complete cluster element</td>
663 <td><code>type=Cluster</code></td>
664 <td><code>type=Cluster,host=${HOST}</code></td>
665 </tr>
666
667 <tr>
668 <td>DeltaManager</td>
669 <td>This manager control the sessions and handle session replication </td>
670 <td><code>type=Manager,context=${APP.CONTEXT.PATH}, host=${HOST}</code></td>
671 <td><code>type=Manager,context=${APP.CONTEXT.PATH}, host=${HOST}</code></td>
672 </tr>
673
674 <tr>
675 <td>FarmWarDeployer</td>
676 <td>Manages the process of deploying an application to all nodes in the cluster</td>
677 <td>Not supported</td>
678 <td><code>type=Cluster, host=${HOST}, component=deployer</code></td>
679 </tr>
680
681 <tr>
682 <td>Member</td>
683 <td>Represents a node in the cluster</td>
684 <td>type=Cluster, component=member, name=${NODE_NAME}</td>
685 <td><code>type=Cluster, host=${HOST}, component=member, name=${NODE_NAME}</code></td>
686 </tr>
687
688 <tr>
689 <td>ReplicationValve</td>
690 <td>This valve control the replication to the backup nodes</td>
691 <td><code>type=Valve,name=ReplicationValve</code></td>
692 <td><code>type=Valve,name=ReplicationValve,host=${HOST}</code></td>
693 </tr>
694
695 <tr>
696 <td>JvmRouteBinderValve</td>
697 <td>This is a cluster fallback valve to change the Session ID to the current tomcat jvmroute.</td>
698 <td><code>type=Valve,name=JvmRouteBinderValve,
699 context=${APP.CONTEXT.PATH}</code></td>
700 <td><code>type=Valve,name=JvmRouteBinderValve,host=${HOST},
701 context=${APP.CONTEXT.PATH}</code></td>
702 </tr>
703
704</table>
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]705</blockquote></td></tr></table><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="FAQ"><strong>FAQ</strong></a></font></td></tr><tr><td><blockquote>
706<p>Please see <a href="http://wiki.apache.org/tomcat/FAQ/Clustering">the clustering section of the FAQ</a>.</p>
刘洪青6266f992017-05-15 21:21:03 +0800[diff] [blame^]707</blockquote></td></tr></table></td></tr><tr class="noPrint"><td width="20%" valign="top" nowrap class="noPrint"></td><td width="80%" valign="top" align="left"><table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#525D76"><font color="#ffffff" face="arial,helvetica.sanserif"><a name="comments_section" id="comments_section"><strong>Comments</strong></a></font></td></tr><tr><td><blockquote><p class="notice"><strong>Notice: </strong>This comments section collects your suggestions
708 on improving documentation for Apache Tomcat.<br><br>
709 If you have trouble and need help, read
710 <a href="http://tomcat.apache.org/findhelp.html">Find Help</a> page
711 and ask your question on the tomcat-users
712 <a href="http://tomcat.apache.org/lists.html">mailing list</a>.
713 Do not ask such questions here. This is not a Q&amp;A section.<br><br>
714 The Apache Comments System is explained <a href="./comments.html">here</a>.
715 Comments may be removed by our moderators if they are either
716 implemented or considered invalid/off-topic.</p><script type="text/javascript"><!--//--><![CDATA[//><!--
717 var comments_shortname = 'tomcat';
718 var comments_identifier = 'http://tomcat.apache.org/tomcat-7.0-doc/cluster-howto.html';
719 (function(w, d) {
720 if (w.location.hostname.toLowerCase() == "tomcat.apache.org") {
721 d.write('<div id="comments_thread"><\/div>');
722 var s = d.createElement('script');
723 s.type = 'text/javascript';
724 s.async = true;
725 s.src = 'https://comments.apache.org/show_comments.lua?site=' + comments_shortname + '&page=' + comments_identifier;
726 (d.getElementsByTagName('head')[0] || d.getElementsByTagName('body')[0]).appendChild(s);
727 }
728 else {
729 d.write('<div id="comments_thread"><strong>Comments are disabled for this page at the moment.<\/strong><\/div>');
730 }
731 })(window, document);
732 //--><!]]></script></blockquote></td></tr></table></td></tr><!--FOOTER SEPARATOR--><tr><td colspan="2"><hr noshade size="1"></td></tr><!--PAGE FOOTER--><tr><td colspan="2"><div align="center"><font color="#525D76" size="-1"><em>
733 Copyright &copy; 1999-2017, Apache Software Foundation
Hongqing Liufd5ee812014-05-10 16:32:51 +0800[diff] [blame]734 </em></font></div></td></tr></table></body></html>