我遇到了大卷曲问题.
我明白了
- nread <= 0,server closed connection,bailing
- transfer closed with outstanding read data remaining
并且内容部分传递
GET /stats/?stats_breakdown=track__track&campaign=&search_criteria=2&period=0&date_month=11&date_day=03&date_year=2015&start_date_month=11&start_date_day=03&start_date_year=2015&end_date_month=12&end_date_day=31&end_date_year=2014 HTTP/1.1 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13 Host: domain.com Accept: */* Cookie: sessionid=xxg4gglsm7o3b224wihqz8od19wl31h1; csrftoken=JBpLxNtgAVvDEw2wNqvBnRmzDJIjxL6C Cache-Control: no-cache Connection: Keep-Alive Keep-Alive: 600 Accept-Language: en-us X-CSrftoken: SeN9bHryRK8FWLTLJIs5c6u9AZ47a8pR Content-Type: application/x-www-form-urlencoded Origin: https://domain.com Referer: https://domain.com * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * additional stuff not fine transfer.c:1037: 0 0 * HTTP 1.1 or later with persistent connection,pipelining supported < HTTP/1.1 200 OK < Server: Nginx/1.8.0 < Date: Wed,04 Nov 2015 12:54:05 GMT < Content-Type: text/html; charset=utf-8 < transfer-encoding: chunked < Connection: keep-alive < vary: Accept-Encoding < vary: Cookie,Accept-Language < P3P: CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI" < Content-Language: en * Replaced cookie csrftoken="JBpLxNtgAVvDEw2wNqvBnRmzDJIjxL6C" for domain domain.com,path /,expire 1478091245 < Set-Cookie: csrftoken=JBpLxNtgAVvDEw2wNqvBnRmzDJIjxL6C; expires=Wed,02-Nov-2016 12:54:05 GMT; Max-Age=31449600; Path=/; secure < * nread <= 0,bailing * transfer closed with outstanding read data remaining * Closing connection #0
这是我使用的PHP配置
function getHeaders()
{
$headers = array();
$headers[] = 'Cache-Control: no-cache';
$headers[] = 'Connection: Keep-Alive';
$headers[] = 'Keep-Alive: 600';
$headers[] = 'Accept-Language: en-us';
$headers[] = 'X-CSrftoken: SeN9bHryRK8FWLTLJIs5c6u9AZ47a8pR';
$headers[] = 'Content-Type: application/x-www-form-urlencoded';
$headers[] = 'Origin: https://domain.com';
$headers[] = 'Referer: https://domain.com';
return $headers;
}
curl_setopt($connection,CURLOPT_URL,$url);
curl_setopt($connection,CURLOPT_RETURNTRANSFER,1);
curl_setopt($connection,CURLOPT_FOLLOWLOCATION,true);
curl_setopt($connection,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
curl_setopt($connection,CURLOPT_SSL_VERIFYPEER,false);
curl_setopt($connection,CURLOPT_SSL_VERIFYHOST,CURLOPT_COOKIEFILE,'cookie.txt');
curl_setopt($connection,CURLOPT_COOKIEJAR,CURLOPT_CONNECTTIMEOUT,550000000);
curl_setopt($connection,CURLOPT_TIMEOUT,5500000000); //timeout in seconds
curl_setopt($connection,CURLOPT_HTTPHEADER,getHeaders());
curl_setopt($connection,CURLOPT_VERBOSE,1);
好吧,经过一些搜索和IRC聊天后,我找到了解决方案,但并不是100％确定原因是什么.看起来保持活动的发送不足以保持连接正常进行.将在此处发布解决方案,希望我可以帮助某人.
对我有帮助的是补充
--keepalive-time 2
卷曲选项的解释
--keepalive-time <seconds>
This option sets the time a connection needs to remain idle before
sending keepalive probes and the time between individual keepalive
probes. It is currently effective on operating systems offering the
TCP_KEEPIDLE and TCP_KEEPINTVL socket options (meaning Linux,recent
AIX,HP-UX and more). This option has no effect if –no-keepalive is
used. (Added in 7.18.0)If this option is used several times,the last one will be used. If
unspecified,the option defaults to 60 seconds.
看起来默认值太高,无法保持连接打开.
这是我用来打电话的完整命令
curl URL -H 'Accept-Encoding: gzip,deflate,sdch' -H 'Accept-Language: en-US,en;q=0.8,et;q=0.6,nl;q=0.4' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_0) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/45.0.2454.101 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Connection: keep-alive' --compressed -v --keepalive-time 2
而且我在osx上运行这个版本的curl
curl 7.43.0 (x86_64-apple-darwin15.0) libcurl/7.43.0 SecureTransport zlib/1.2.5 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp Features: AsynchDNS IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz UnixSockets
如果有人想在PHP curl中使用此选项,则自PHP 5.5起可以使用–keepalive-time选项.您可以按如下方式使用它：
curl_setopt($connection,CURLOPT_TCP_KEEPALIVE,1); curl_setopt($connection,CURLOPT_TCP_KEEPIDLE,2);
希望这可以帮助有人在同样的问题上挣扎！