Quick Tip: HTTP GET with the ESP8266 Thing

By:  September 16, 2015 Categories:  Tags: , , , , 

The SparkFun ESP8266 “Thing” is one of the cheapest Internet of Things (IoT) platforms available.

SparkFun ESP8266 Thing
Photo courtesy of sparkfun.com

There are some great examples on how to post data to data.sparkfun.com, but we need to modify that code in order to pull data from a website. That is accomplished with the humble HTTP GET request. I put together a quick example that pulls www.example.com and prints it to the serial console. Note that you will need perform some slight modifications to the board (or use Realterm) to see the serial data.

[Updated 11/26/15] Added #include <WiFiClient.h>. Thanks, Naguissa!

#include <ESP8266WiFi.h>
#include <WiFiClient.h>

// WiFi information
const char WIFI_SSID[] = "My_WiFi_SSID";
const char WIFI_PSK[] = "My_WiFi_Password";

// Remote site information
const char http_site[] = "www.example.com";
const int http_port = 80;

// Pin definitions
const int LED_PIN = 5;

// Global variables
WiFiClient client;

void setup() {
  
  // Set up serial console to read web page
  Serial.begin(9600);
  Serial.print("Thing GET Example");
  
  // Set up LED for debugging
  pinMode(LED_PIN, OUTPUT);
  
  // Connect to WiFi
  connectWiFi();
  
  // Attempt to connect to website
  if ( !getPage() ) {
    Serial.println("GET request failed");
  }
}

void loop() {
  
  // If there are incoming bytes, print them
  if ( client.available() ) {
    char c = client.read();
    Serial.print(c);
  }
  
  // If the server has disconnected, stop the client and WiFi
  if ( !client.connected() ) {
    Serial.println();
    
    // Close socket and wait for disconnect from WiFi
    client.stop();
    if ( WiFi.status() != WL_DISCONNECTED ) {
      WiFi.disconnect();
    }
    
    // Turn off LED
    digitalWrite(LED_PIN, LOW);
    
    // Do nothing
    Serial.println("Finished Thing GET test");
    while(true){
      delay(1000);
    }
  }
}

// Attempt to connect to WiFi
void connectWiFi() {
  
  byte led_status = 0;
  
  // Set WiFi mode to station (client)
  WiFi.mode(WIFI_STA);
  
  // Initiate connection with SSID and PSK
  WiFi.begin(WIFI_SSID, WIFI_PSK);
  
  // Blink LED while we wait for WiFi connection
  while ( WiFi.status() != WL_CONNECTED ) {
    digitalWrite(LED_PIN, led_status);
    led_status ^= 0x01;
    delay(100);
  }
  
  // Turn LED on when we are connected
  digitalWrite(LED_PIN, HIGH);
}

// Perform an HTTP GET request to a remote page
bool getPage() {
  
  // Attempt to make a connection to the remote server
  if ( !client.connect(http_site, http_port) ) {
    return false;
  }
  
  // Make an HTTP GET request
  client.println("GET /index.html HTTP/1.1");
  client.print("Host: ");
  client.println(http_site);
  client.println("Connection: close");
  client.println();
  
  return true;
}

 

62 thoughts on “Quick Tip: HTTP GET with the ESP8266 Thing”

  • Hello,

    Thanks for the post. I’m doing a home alarm system and using a GET request to my server to raise alerts. This example is perfect as base to the program.

    One detail: There’re two libraries missing needed to run the sketch:

    #include
    #include

    Cheers.

    Reply

  • what am I careless!
    thank you very much naguissa it works..
    “”Line 96: client.println(“GET /example HTTP/1.1”);””

    Reply

  • My webpage is returning JSON which gets printed to the serial console by this section:

    // If there are incoming bytes, print them
    if ( client.available() ) {
    char c = client.read();
    Serial.print(c);
    }

    How can I store this into a variable so that I can use it with a JSON parsing library? I’ve been struggling with this for days…

    Reply

    • You will need to create a buffer (probably a byte or char array) and store everything you get from client.read(). You will also need to look for an “End of File” or “End of Stream” character (or set of characters) to let the loop know when you’re done. I’m not sure what this would be from your webpage, so you’ll have to look at the output to determine that. A good example of how this works is the readPage() function in this example: http://bildr.org/2011/06/arduino-ethernet-client/.

      Reply

  • my module(olimex esp8266 dev board) LED still blinking not connected to WiFi after upload the sketch several times.

    Reply

  • What if I want to set a cookie before accessing a webpage.. That’s because the webpage redirects to some other page without that cookie..

    Reply

  • How can I read out the source code of a website? or webdokument
    I have a txt-file on my server with temperature data which I want to display via a esp8266

    Reply

    • If you are using a browser on a desktop, you can press ‘F12’ to view the source (HTML). Are you using the ESP8266 as the server, or is the ESP8266 trying to read the text file from another computer?

      Reply

      • Yes the ESP8266 is trying to read a text file from a webserver data.MYDOMAIN.com/data.txt

        The text file just look like this:
        0,255,255,0

        Reply

        • From a browser on another computer, can you navigate to the site “data.MYDOMAIN.com/data.txt” and see the contents in the browser window? This will let you know if the server is responding to GET requests.

          Reply

  • What method would I use to send a variable to a php file using this code?
    I have tried the same way I do with the regular Ethernet.h library but nothing is showing up in my database…
    client.print( “GET /add_data_motion1.php?”);
    client.print(“motion1=”);
    client.print( motion1 );
    client.println( ” HTTP/1.1″);
    client.print( “Host: ” );
    client.println(http_site);
    client.println( “Connection: close” );
    client.println();

    Thanks!

    Reply

  • I want to fetch the JSON data from Node.Js Server to ESP8266-01 using http GET method.

    I have used Software serial definition as
    #include
    SoftwareSerial ser(A8,A9); // Connect TX and RX of ESP8266 (A8 for TX A9 for RX) of ATMEGA2560.

    In the code i have written

    ser.print(String(“GET “) + “/details HTTP/1.1/\r\n” +
    “192.168.1.11”+ “\r\n” +
    “Connection: close\r\n\r\n”);

    // Read all the lines of the reply from server and print them to Serial
    Serial.println(“Respond:”);
    while(ser.available()){
    String line = ser.readStringUntil(‘\r’);
    Serial.print(line);
    }

    Serial.println();
    Serial.println(“closing connection”);

    where ser is defined function for ESP8266-01
    192.168.1.11 is Local server IP address from where i have to fetch the JSON data(data and time in my case).

    But i am not able to fetch date and time data.

    Reply

    • Are you able to access the JSON data from another client, like your computer?

      Additionally, I’m not super well versed in Arduino’s String object. Try doing a Serial.print with your entire GET request to make sure it looks right. I get the feeling trying to concatenate a String object with strings (arrays of characters) might not work that well.

      Reply

      • Yes.I am able to access data from another client.

        Sir ,i will post my query again in more understandable form.

        We have created Node.Js server in our office network for testing purpose.

        Now i am able to send the data to Node.js server that is temperature and humidity values are getting updated to Node.js server.

        Now i am stucked with receiving the data from Node.Js server(date and time from internet ) to ESp8266 and have to dispay on serial monitor of arduino which is not happening .

        Here is the code.

        I have initialized for ESP8266 for ATMEGA 2560 as below shown code and

        used ser.println function,hope using “ser” is correct? 

        #include 

        SoftwareSerial ser(A8,A9);

        Then in the esp8266 function as

        void esp_8266()
        {

        String cmd = “AT+CIPSTART=\”TCP\”,\””; // AT cammand for TCP/IP connection
        cmd += “192.168.1.11”; // node.js Server Ip address
        cmd += “\”,5000″; // HTTP port no.

        ser.println(cmd); // Sending AT cammand
        p=1;
        if(p == 1)
        {
        if(ser.find(“Linked”)) // Serching for responce from ESP8266
        {

        Serial.println(“Connection Made for Receiving data”);
        z=0;
        }

        p=0;
        }

        int deviceid = serialNumber;
        String getStr = “GET /log/345/12/status/”; // /log is the url , 345,12 ,status are parameters which are hardcoded for testing

        getStr += DHT.temperature; //room temperature
        getStr += “/”;
        getStr += DHT.humidity; //room humidity
        getStr += “\r\n\r\n”;

        cmd = “AT+CIPSEND=”; // AT command to send Data at TCP server
        cmd += String(getStr.length()); // send data length
        ser.println(cmd); // Wrighting the AT command to ESP8622

        if(ser.find(“>”)) // Serching for responce to send data from ESP8266
        {

        ser.print(getStr); // Printing the string with sensor data to send on ESP8266 module

        }

        delay (100);

        ser.print(String(“GET “) + “/details HTTP/1.1/\r\n” +
        “192.168.1.11”+ “\r\n” +
        “Connection: close\r\n\r\n”);

        // Read all the lines of the reply from server and print them to Serial
        Serial.println(“Respond:”);
        while(ser.available()){
        String line = ser.readStringUntil(‘\r’);
        Serial.println(line);
        }

        Serial.println();
        Serial.println(“closing connection”);
        }

        Reply

      • hey man, I’m getting this error below. Any idea why?

        In file included from C:\Users\01738\Documents\Arduino\libraries\ESP8266WiFi\src/ESP8266WiFi.h:33:0,

        from C:\Users\01738\Documents\Arduino\ESP8266_ATfromcode2\ESP8266_ATfromcode2.ino:9:

        C:\Users\01738\Documents\Arduino\libraries\ESP8266WiFi\src/ESP8266WiFiType.h:26:19: fatal error: queue.h: No such file or directory

        #include

        ^

        compilation terminated.

        Using library ESP8266WiFi at version 1.0 in folder: C:\Users\01738\Documents\Arduino\libraries\ESP8266WiFi
        exit status 1
        Error compiling for board Arduino/Genuino Mega or Mega 2560.

        Reply

    • That seems odd since it’s a fairly basic HTTP page, no SSL or anything. Do you have any more information on the error, such as an HTTP status code? Can you successfully request other non-SSL sites?

      Reply

      • Thanks for responding! I’ve tried other sites e.g. http://www.google.com with success as well as http://www.example.com. However this site http://api.usno.navy.mil/easter?year=2016 or this usefull site api.usno.navy.mil/rstt/oneday?date=9/14/2016&coords=47.73N,9.16E&tz=1 respond to an browser request but don’t accept a connection from within the sample program. May be you find the time to test it by your own.Thank you for supporting me.

        here is my minor modifyed code (line 9-12, ine 97ff:
        #include
        #include

        // WiFi information
        const char WIFI_SSID[] = “—“;
        const char WIFI_PSK[] = “—-“;

        // Remote site information
        //const char http_site[] = “www.example.com”;
        const char http_site[] = “http://api.usno.navy.mil/rstt/oneday?date=9/14/2016&coords=47.73N,9.16E&tz=1”; // this one don’t respond
        const char site[] = “http://api.usno.navy.mil/easter?year=2016”; // this one don’t respond -> no server connection however as browser request it works fine!
        //char site[] = “www.google.com”; // this one works fine, “got connection”

        const int http_port = 80;

        const int LED_PIN = 16; //16=red LED 2=blue LED on ESP-12 board

        // Global variables
        WiFiClient client;

        void setup() {

        // Set up serial console to read web page
        Serial.begin(115200);
        Serial.println(); Serial.println();
        Serial.println(“Thing GET Example”);

        // Set up LED for debugging
        pinMode(LED_PIN, OUTPUT);

        // Connect to WiFi
        connectWiFi();

        // Attempt to connect to website
        if ( !getPage() ) {
        Serial.println(“GET request failed”);
        }
        }

        void loop() {

        // If there are incoming bytes, print them
        if ( client.available() ) {
        char c = client.read();
        Serial.print(c);
        }

        // If the server has disconnected, stop the client and WiFi
        if ( !client.connected() ) {
        Serial.println();

        // Close socket and wait for disconnect from WiFi
        client.stop();
        if ( WiFi.status() != WL_DISCONNECTED ) {
        WiFi.disconnect();
        }

        // Turn off LED
        digitalWrite(LED_PIN, LOW);

        // Do nothing
        Serial.println(“Finished Thing GET test”);
        while(true){
        delay(1000);
        }
        }
        }

        // *************** Attempt to connect to WiFi
        void connectWiFi() {

        byte led_status = 0;

        // Set WiFi mode to station (client)
        WiFi.mode(WIFI_STA);

        // Initiate connection with SSID and PSK
        WiFi.begin(WIFI_SSID, WIFI_PSK);

        // Blink LED while we wait for WiFi connection
        while ( WiFi.status() != WL_CONNECTED ) {
        digitalWrite(LED_PIN, led_status);
        led_status ^= 0x01;
        delay(100);
        }

        // Turn LED on when we are connected
        digitalWrite(LED_PIN, HIGH);
        Serial.println(“WiFi Connected”);
        }

        // ************* Perform a HTTP GET request to a remote page
        bool getPage() {

        // Attempt to make a connection to the remote server
        // if ( !client.connect(http_site, http_port) ) {
        delay(500);
        if ( client.connect(site, http_port))
        {Serial.println(“server connected”);
        return true;}
        else
        {
        Serial.println(“no server connection”);
        return false;}

        Reply

        • Alright, I got it working. As it turns out, you can’t list the full URL in http_site. It should just be the base site, “api.usno.navy.mil/” (no http://) otherwise the DNS server won’t return an IP address. You then need to add the resource location “/rstt/oneday?date=9/14/2016&coords=47.73N,9.16E&tz=1” right after “GET.” I don’t have a standalone ESP8266, so I couldn’t use your code exactly. Here is what I used on the SparkFun Thing:

          Edit: Code formatting doesn’t seem to work in comments, so I posted it here: https://gist.github.com/ShawnHymel/3b1e2d2649f5dded8f66ed1e9ff8d35c

          Reply

  • Thanks for the example and support! May I ask for help again.
    While setup WiFi connection my scetch – based on your example (thanks again!) – occasionally (app. every third call) end with an Exception (9) …
    I’ve no idea what’s the reason nor how to avoid this :-((
    ——
    char versions_id[] = “ESP__WEB_test_V6”;
    #include
    #include
    #include
    #include

    const char WIFI_SSID[] = “***”; // WiFi information
    const char WIFI_PSK[] = “***”;

    const char http_site[] = “api.usno.navy.mil”; // Remote site information
    const int http_port = 80;

    const int LED_PIN = 16; // global variables & definitions
    WiFiClient client;
    char c; // incomming char from WEB-server
    unsigned long delayMillis; // delay loop time
    unsigned long pause = 20000; // prog paused

    // JSON stuff
    StaticJsonBuffer jsonBuffer; // Memory pool for JSON object tree.
    char json[500]; // JSON object char array, filld with server response
    String head = “”; // HTTP header from server
    int json_i=0; // index for json char array
    int JS_bracket = 0; // counter for JSON { and } brackets

    void setup() {
    Serial.begin(115200); // Set up serial console to read web page
    Serial.println(); Serial.println(versions_id);
    Serial.println(“********************RE-/START******************”);
    pinMode(LED_PIN, OUTPUT); // Set up LED for debugging
    }

    void loop() {
    head = “”; // reset server response string
    JS_bracket = 0; // reset json bracket counter

    connectWiFi(); // Connect to WiFi

    if ( !getPage() ) { // Attempt to connect to website
    Serial.println(“GET request failed”);}
    else {
    Serial.println(“GET request ok”);}

    while (client.connected() ) {
    if ( client.available() ) { // If there are incoming bytes, print them
    c = client.read();
    //Serial.print(c);
    if (JS_bracket == 0) {head=head+c;} // fill in server response
    if (c == ‘{‘) {++JS_bracket;} // ectraxt json object
    if (c == ‘}’) {–JS_bracket; }
    // still to do: skip white space outside “..”
    if (JS_bracket > 0) { //Serial.print(c); // look for matching closing bracket
    json[json_i]=c; json_i++;}; }
    }
    json[json_i]=’}’; // add a closing } bracket
    Serial.println(“stop client”); // If the server has disconnected, stop the client and WiFi
    client.stop(); // Close socket and wait for disconnect from WiFi
    if ( WiFi.status() != WL_DISCONNECTED ) {
    WiFi.disconnect();}
    digitalWrite(LED_PIN, HIGH); // Turn off LED

    // *****process server data here
    Serial.println(“processing server data”);
    // for (int i=0; i<=json_i; i++) {Serial.print(json[i]);} Serial.println(); // control print
    Serial.println("******************pausing****************");
    delayMillis= millis()+pause; // 5 minutes intervall
    do {
    yield();
    } while (millis() < delayMillis);
    Serial.println("***********start after pausing***********");
    // *****end processing server data

    }

    //******************connect WiFi***********************
    void connectWiFi() { // connect WiFi
    byte led_status = 0;
    WiFi.mode(WIFI_STA); // Set WiFi mode to station (client)
    WiFi.begin(WIFI_SSID, WIFI_PSK); // Initiate connection with SSID and PSK
    while ( WiFi.status() != WL_CONNECTED ) { // Blink LED while we wait for WiFi connection
    digitalWrite(LED_PIN, led_status);
    led_status ^= 0x01;
    delay(100);
    }
    digitalWrite(LED_PIN, HIGH); // Turn LED on when we are connected
    }

    //***********************get page***************
    bool getPage() { // Perform an HTTP GET request to a remote page
    if ( !client.connect(http_site, http_port) ) { // Attempt to make a connection to the remote server
    return false;
    }
    client.println("GET /rstt/oneday?date=9/14/2016&coords=47.73N,9.16E&tz=1 HTTP/1.1"); // Make an HTTP GET request
    client.print("Host: ");
    client.println(http_site);
    client.println("Connection: close");
    client.println();
    return true;
    }

    Reply

  • some one eats text inside angle brackets

    #include ..ESP8266WiFi.h..
    #include ..WiFiClient.h..
    #include ..ArduinoJson.h..
    #include ..TimeLib.h..

    Reply

  • Hi to all

    I´m newbie in arduino and ndemcu, and I have a question.

    I´m testing my nodemcu esp8266 1.0 esp-12E module module with arduino ide and I already made some little projects and all it´s working fine.

    I would like to know if its possible to access a url from nodemcu , i.e.,
    I have a account in voipraider thats allow me to send a sms through this url:

    If I paste this url in any browser, voipraider send a sms to destination_phone.

    My goal is create a sketch in nodemcu to send a sms using this url.

    Its possible ?
    If yes, could you please help me with some code to use in arduino ide (not LUA) ?

    I would be very grateful if you could help me.

    Thanks for your attention

    Reply

    • I’m sorry, but I am not familiar with NodeMCU, so I don’t think I’ll be able to help you much. If you are just pasting a URL into a browser, then you are making an HTTP GET request. As long as you can do that with your microcontroller, then you should (in theory) be able to make the SMS request.

      Reply

  • Hi to all

    I´m newbie in arduino and ndemcu, and I have a question.

    I´m testing my nodemcu esp8266 1.0 esp-12E module module with arduino ide and I already made some little projects and all it´s working fine.

    I would like to know if its possible to access a url from nodemcu , i.e.,
    I have a account in voipraider thats allow me to send a sms through this url:

    If I paste this url in any browser, voipraider send a sms to destination_phone.

    My goal is create a sketch in nodemcu to send a sms using this url:

    Its possible ?
    If yes, could you please help me with some code to use in arduino ide (not LUA) ?

    I would be very grateful if you could help me.

    Thanks for your attention

    Reply
  • Pingback: دریافت اطلاعات از سرور با ماژو esp8266

  • Thanks for that. It saved a lot of time and ultimately is very simple. I am using it as the basis to record sensor data on my own cloud. I haven’t had much luck with analog sensors on the ESP8266 but still useful for PIR sensors etc.

    Reply

  • Is it possible to make a http get request per second via esp8266??
    The response time is slow. I want to upload sensor data second by second. Is there any method to do it? Sir, Please help!!

    Reply

    • You should be able to make an HTTP request once per second with the ESP8266. However, several factors may prevent getting a response every second. For example, your internet connection may be slow (try pinging your destination server to see if you get a response time of >1 s). If you’re trying to receive more than a few bytes as an HTTP response, the ESP8266 is likely too slow to process it. Additionally, if you are using a site like ThingSpeak to log your data, you are limited to posting once every 15 seconds, so make sure you read the fine print in whichever service you are using.

      Reply

  • Hi ShawnHymel 🙂
    How can I send GET/POST from my website using php and receive data on my ESP ?

    Reply

  • I configured my router to do portforwarding, and it works perfect 🙂 I received data on ESP from my website. I used JQuery library in my website to send HTTP GET request to the IP address(ESP).

    Reply

  • how can do it loop forever??
    —————————————-
    #include
    #include

    // WiFi information
    const char WIFI_SSID[] = “My_WiFi_SSID”;
    const char WIFI_PSK[] = “My_WiFi_Password”;

    // Remote site information
    const char http_site[] = “www.example.com”;
    const int http_port = 80;

    // Pin definitions
    const int LED_PIN = 5;

    // Global variables
    WiFiClient client;

    void setup() {

    // Set up serial console to read web page
    Serial.begin(9600);
    Serial.print(“Thing GET Example”);

    // Set up LED for debugging
    pinMode(LED_PIN, OUTPUT);

    // Connect to WiFi
    connectWiFi();

    // Attempt to connect to website
    if ( !getPage() ) {
    Serial.println(“GET request failed”);
    }
    }

    void loop() {

    // If there are incoming bytes, print them
    if ( client.available() ) {
    char c = client.read();
    Serial.print(c);
    }

    // If the server has disconnected, stop the client and WiFi
    if ( !client.connected() ) {
    Serial.println();

    // Close socket and wait for disconnect from WiFi
    client.stop();
    if ( WiFi.status() != WL_DISCONNECTED ) {
    WiFi.disconnect();
    }

    // Turn off LED
    digitalWrite(LED_PIN, LOW);

    // Do nothing
    Serial.println(“Finished Thing GET test”);
    while(true){
    delay(1000);
    }
    }
    }

    // Attempt to connect to WiFi
    void connectWiFi() {

    byte led_status = 0;

    // Set WiFi mode to station (client)
    WiFi.mode(WIFI_STA);

    // Initiate connection with SSID and PSK
    WiFi.begin(WIFI_SSID, WIFI_PSK);

    // Blink LED while we wait for WiFi connection
    while ( WiFi.status() != WL_CONNECTED ) {
    digitalWrite(LED_PIN, led_status);
    led_status ^= 0x01;
    delay(100);
    }

    // Turn LED on when we are connected
    digitalWrite(LED_PIN, HIGH);
    }

    // Perform an HTTP GET request to a remote page
    bool getPage() {

    // Attempt to make a connection to the remote server
    if ( !client.connect(http_site, http_port) ) {
    return false;
    }

    // Make an HTTP GET request
    client.println(“GET /index.html HTTP/1.1”);
    client.print(“Host: “);
    client.println(http_site);
    client.println(“Connection: close”);
    client.println();

    return true;
    }

    Reply

    • Sorry that I missed this last month! The loop() function is found in the Arduino framework. If you dig through Arduino’s source code, you’ll find a main() that looks something like:

      void main() {
  setup();
  while(1) {
    loop();
  }
}

      So the main() is hidden from users, who just need to implement setup() and loop() functions.

      Reply
  • Pingback: URL

Leave a Reply

Your email address will not be published. Required fields are marked *