wget - Robot names for robots.txt -

-

suppose have website uses wget crawl other websites. provide website owners chance of not being crawled website. should use robot name wget in robots.txt file, or have create other name?

common practice disallow , allow popular uas this:

user-agent: google disallow:   user-agent: * disallow: /

so think don't have problems using wget way


Comments

Post a Comment

Popular posts from this blog

php - Array to string conversion smarty template -

-
hello have array string conversion erro code: index.php: code: $hoarr = array(1,2,3); foreach ($hoarr $hid) { $mysqli = mysqli_connect("localhost", "root", "", "zabbixtest"); if (!$mysqli) { die("connection failed: " . mysqli_connect_error()); } $sql = "select errors_from hosts hostid = '$hid'"; $re = mysqli_query($mysqli, $sql); while ($row = mysqli_fetch_assoc($re)) { $query_times[] = $row['errors_from']; } mysqli_free_result($re); mysqli_close($mysqli); } $smarty->assign('query_times',$query_times); and index.tpl: code: <p>{$query_times}</p> i have no idea whats wrong it.. you try assign array $smarty->assign can assign string. use instead: $query_string = implode ( "," , $query_times ); $smarty->assign('query_times',$query_string);
Read more

Winapi c++: DialogBox hangs when breaking a loop -

-
i've created s simple w32 application cicle through 2 applications(hard coded window classes atm). when call start button (idc_start) works fine, when change focus hwnd application hangs cannot closed. need simple , clean method stop loop, start when calling idc_start. apreciated! #include <stdio.h> #include <stdlib.h> #include <conio.h> #include <windows.h> #include <resource.h> #include <iostream> using namespace std; bool callback dlgproc(hwnd hwnd, uint message, wparam wparam, lparam lparam) { hwnd hwnd1,hwnd2; hwnd1 = findwindow("notepad++",0); hwnd2 = findwindow("notepad",0); bool bdone; switch(message) { case wm_initdialog: // set dialog box, , initialise default values { setdlgitemint(hwnd, idc_number, 5, false); hicon hicon, hiconsm; hicon = (hicon)loadimage(null, "e32.ico", image_icon, 32, 32, lr_loadfromfile); if(hicon) se...
Read more

objective c - Can't build GCM with Protobuf in Xcode -

-
recently i've decided integrate gcm project uses google protocol buffers. worked fine, sample ok. when i've tried integrate gcm via cocoapods project - started receiving build errors: undefined symbols architecture x86_64: "_gsdk_pblogicalrightshift32", referenced from: _serializedsize in libgcmlib.a(gcmsecuresocket.o) "_objc_class_$_gsdk_pbcodedoutputstream", referenced from: objc-class-ref in libgcmlib.a(gcmsecuresocket.o) "_objc_class_$_gsdk_pbdescriptor", referenced from: objc-class-ref in libgcmlib.a(gtalkcore.pb.o) objc-class-ref in libgcmlib.a(gtalkextensions.pb.o) "_objc_class_$_gsdk_pbgeneratedmessage", referenced from: _objc_class_$_gtalkheartbeatping in libgcmlib.a(gtalkcore.pb.o) _objc_class_$_gtalkheartbeatack in libgcmlib.a(gtalkcore.pb.o) _objc_class_$_gtalkerrorinfo in libgcmlib.a(gtalkcore.pb.o) _objc_class_$_gtalksetting in libgcmlib.a(gtalkcore.pb.o) _objc_class_$_gtalkloginrequest in libgcmlib.a...
Read more