一个简单的HttpClient多线程工具类

近期发现某项目运行一段时间后,端口占用非常多,导致程序运行异常。

经查发现里面有使用HttpClient的方式如下:

HttpClient client = new HttpClient();

…..

client.executeMethod(method)

该代码在服务器不主动关闭连接的情况下会发起大量的网络请求导致资源耗尽。

大部分人使用HttpClient都是使用类似上面的代码,包括Apache官方的例子也是如此。

具体大家可以移步:http://seanhe.iteye.com/blog/234759 作者做了较详细的分析。

考虑到接口的是定期获取采集数据的,访问比较频繁,连接也无需关闭,顺手写了一个简单HttpClient连接池。

package com.stock.util;

import java.util.Queue;
import java.util.concurrent.LinkedBlockingDeque;

import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.methods.GetMethod;

 

/**
 * 多线程HTTP 调用工具类 <br>
 * Date: 2015-7-15
 *
 * @author snow2back
 * @version 1.0
 *
 */
public class HttpUtil {
	private static Queue<HttpClient> clientQueue = new LinkedBlockingDeque<HttpClient>();

	private static int queueSize = 0;

	/**
	 * 最大连接数
	 **/
	private static final int MAX_QUEUE_SIZE = 3;

	static ThreadLocal<HttpClient> local = new ThreadLocal<HttpClient>();

	public static HttpClient getClient() {
		// System.out.println(Thread.currentThread().getId()+"获取httpclient");
		HttpClient client = null;
		synchronized (clientQueue) {
			client = local.get();
			if (client == null) {
				// 获取一个
				if (clientQueue.size() > 0) {
					client = clientQueue.poll();
				} else {
					if (queueSize <= MAX_QUEUE_SIZE) {
						queueSize++;
						client = new HttpClient();
						client.getHttpConnectionManager().getParams()
						        .setSoTimeout(5000);
						client.getHttpConnectionManager().getParams()
						        .setConnectionTimeout(10000);
					} else {
						try {
							clientQueue.wait();
						} catch (InterruptedException e) {
							e.printStackTrace();
						}
						client = getClient();
					}
				}
				local.set(client);
			}
		}
		return client;
	}

	public static void release() {
		synchronized (clientQueue) {
			// System.out.println(Thread.currentThread().getId()+"归还httpclient");
			// 归还HTTPCLient
			clientQueue.add(local.get());
			local.remove();
			clientQueue.notify();
		}
	}

	public static final String fetchUrl(String url) {
		String rep = "";
		// ajax 数据抓取
		GetMethod method = new GetMethod(url);
		method.addRequestHeader("Accept",
		        "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
		method.addRequestHeader("Accept-Language:", "zh-CN,zh;q=0.8");
		method.addRequestHeader(
		        "User-Agent",
		        "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.65 Safari/537.36");
		try {
			// 设置http代理
			// getClient().getHostConfiguration().setProxy("10.119.6.6", 8886);
			int status = getClient().executeMethod(method);
			if (status == 200) {
				rep = method.getResponseBodyAsString();
			}
		} catch (Exception e) {
			e.printStackTrace();
		}
		release();
		return rep;
	}

	public static void main(String args[]) {
		String resp = HttpUtil
		        .fetchUrl("http://10.119.0.17/PhdSvervice/TagTime.asmx");
		System.out.println(resp);
	}
}

重新部署,程序稳定运行,问题得到解决~

大家也可以考虑更换默认的ConnectionManager为MultiThreadedHttpConnectionManager,也可以达到一样的效果。

可以看下这篇文章:http://blog.csdn.net/kobejayandy/article/details/16921265