While images can be uploaded to Twitter in one request, uploading a video is a different story.

.NET Core, Twitter

And although I’ve managed to implement it, I am not entirely happy with the result, and later you will see why is that.

Unsolved: While it all works fine, I don't like the fact that chunks are sent as a part of URL query string

Video requirements

Not every video will be accepted by Twitter API - take a look at their specifications. Although, I am not sure if you can trust those, because here it says:

File size must not exceed 512 MB

And on another page it says:

Size restrictions for uploading via API
  • ...
  • Video 15MB

Go figure.

When it comes to other characteristics like codec and others, in my case I needed to make a video out of 2 images, so here’s the FFmpeg command:

ffmpeg -r 1/4 -i "concat:1.png|2.png" -pix_fmt yuv420p video.mp4

The result video file is accepted by Twitter with no problems.

Upload process

Here’re some official guidelines. Eventually you will come to understanding that video uploads need to go through chunked media upload.

Unlike regular upload media single request, chunked upload consists of four requests (or rather stages, because some stages can have more than one request of their own):

  1. INIT
  2. APPEND
  3. FINALIZE
  4. STATUS

Implementation

The following implementation continues/extends the code I had wrote earlier.

INIT

INIT request initializes the upload. If it is successful, you will get a response with media_id identificator, which will be used for all further requests.

public async Task<Tuple<int, string>> SendInit(long videoSizeBytes)
{
    using (var httpClient = new HttpClient())
    {
        var initData = new Dictionary<string, string>
        {
            { "command", "INIT" },
            { "total_bytes", $"{videoSizeBytes}" },
            { "media_type", "video/mp4" },
            { "media_category", "tweet_video" } // not really required
        };

        httpClient.DefaultRequestHeaders.Add(
            "Authorization",
            PrepareOAuth(_TwitterMediaAPI, initData, "POST")
            );
        var httpResponse = await httpClient.PostAsync(
            _TwitterMediaAPI,
            new FormUrlEncodedContent(initData)
            );
        var httpContent = await httpResponse.Content.ReadAsStringAsync();

        return new Tuple<int, string>(
            (int)httpResponse.StatusCode,
            httpContent
            );
    }
}

Note that the media_category parameter is not mandatory, I tried sending requests without it, and my video was posted anyway, so I can’t tell what role does it actually play (documentation says that it “enables advanced features”), but I decided to keep it just in case.

The method can be called like this:

string pathToVideo = "video.mp4";
byte[] videoData = System.IO.File.ReadAllBytes(pathToVideo);

var rezInit = Task.Run(async () =>
{
    var response = await twitter.SendInit(videoData.Length);
    return response;
});

if (rezInit.Result.Item1 != 202)
{
    Console.WriteLine($"[ERROR] {rezInit.Result.Item1}: {rezInit.Result.Item2}");
    return;
}
var rezInitJson = JObject.Parse(rezInit.Result.Item2);

// here's the media_id value we're after
string mediaID = rezInitJson["media_id"].Value<string>();

Just in case, here’s an example of a full JSON response:

{
  "media_id": 1193135308410889857,
  "media_id_string": "1193135308410889857",
  "expires_after_secs": 86399,
  "media_key": "7_1193135308410889857"
}

And the response code should be 202.

APPEND

That was the most difficult part. First of all, chunked upload means that you need to slice your file in portions (chunks) and send those one by one, so you need to organize a chain of requests. That alone was not the easiest thing in the world, but I also got several other problems along the way.

The most troublesome one was this:

Segments do not add up to provided total file size

So, after I sent all the chunks and finished the chain with FINALIZE, Twitter responded that the chunked file I sent has a different size, comparing to the amount of bytes I announced in INIT.

That didn’t makes sense, and I wasted a lot of time looking for a solution. I tried to find how other people did it, but surprisingly there are very few .NET/C# implementations. And one of those is Tweetinvi library - a monumental piece of software, which sources are so complicated, that soon enough I fell into despair and gave up on the idea to get anything useful out of it.

But eventually I found out what was wrong. It turned out that I haven’t actually sent a single chunk, because all my APPEND requests were failing! And I didn’t notice it, because I wasn’t checking if responses contain any errors. That simple.

And the error in my case was failed authentication:

{
  "errors": [
    {
      "code": 32,
      "message": "Could not authenticate you."
    }
  ]
}

That happened because first I tried to use the same authentication scheme as with sending images:

httpClient.DefaultRequestHeaders.Add(
    "Authorization",
    PrepareOAuth(_TwitterMediaAPI, null, "POST")
    );

Because why would I do it otherwise, right? That’s the way it works for sending images. Besides, documentation says so as well:

Because the method uses multipart POST, OAuth is handled differently. POST or query string parameters are not used when calculating an OAuth signature basestring or signature. Only the oauth_* parameters are used.

Nevertheless, my requests were failing till I added query string parameters to the signature as well.

Here’s the method’s code:

public async Task<Tuple<int, string>> SendAppend(byte[] chunk, int chunkID, string mediaID)
{
    using (var httpClient = new HttpClient())
    {
        // chunks need to be base64-encoded
        string videoBase64 = Convert.ToBase64String(chunk);

        var appendData = new Dictionary<string, string>
        {
            { "command", "APPEND" },
            { "media_id", $"{mediaID}" },
            { "media_data", $"{videoBase64}" },
            { "segment_index", $"{chunkID}" }
        };

        var fuec = new FormUrlEncodedContent(appendData);
        fuec.Headers.ContentType = new MediaTypeHeaderValue(
            "application/x-www-form-urlencoded"
            );

        httpClient.DefaultRequestHeaders.Add(
            "Authorization",
            PrepareOAuth(_TwitterMediaAPI, appendData, "POST")
            );
        var httpResponse = await httpClient.PostAsync(_TwitterMediaAPI, fuec);
        var httpContent = await httpResponse.Content.ReadAsStringAsync();

        return new Tuple<int, string>(
            (int)httpResponse.StatusCode,
            httpContent
            );
    }
}

As you can see, I set application/x-www-form-urlencoded in Content-Type header. Even though documentation says:

Requests should be either multipart/form-data or application/x-www-form-urlencoded POST formats.

in my case multipart/form-data header did not work. It actually did not work in two ways: if query parameters are signed too, then it’s this again:

Could not authenticate you.

and if I pass null, like when sending images, then it’s something different:

{
    "errors": [
        {
            "code": 214,
            "message": "Bad request."
        }
    ]
}

And application/x-www-form-urlencoded worked just fine.

Speaking about failing authentication, since I am using FormUrlEncodedContent instead of MultipartFormDataContent, apparently that’s why query parameters have to be signed as well.

Here’s how to use the method:

const int chunkSize = 40 * 1024; // read the file by chunks of 40 KB
using (var file = File.OpenRead(pathToVideo))
{
    int bytesRead, chunkID = 0;
    var buffer = new byte[chunkSize];
    while ((bytesRead = file.Read(buffer, 0, buffer.Length)) > 0)
    {
        if (bytesRead < chunkSize)
        {
            var lastBuffer = new byte[bytesRead];
            Buffer.BlockCopy(buffer, 0, lastBuffer, 0, bytesRead);
            buffer = new byte[bytesRead];
            buffer = lastBuffer;
        }
        try
        {
            var rezAppend = Task.Run(async () =>
            {
                var response = await twitter.SendAppend(
                    buffer,
                    chunkID,
                    mediaID
                    );
                return response;
            });
            if (rezAppend.Result.Item1 != 204)
            {
                Console.WriteLine($"[ERROR] {rezAppend.Result.Item1}: {rezAppend.Result.Item2}");
                return;
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine($"[ERROR] {ex.Message}");
            return;
        }
        chunkID++;
    }
}

So the method is called in a loop till the whole file is read.

To each APPEND request Twitter should respond with 204 code. There is JSON response, only this code.

Now we get to the point which I mentioned in the very beginning - that I am not entirely happy with my implementation. So the problem here is that by the looks of it, file chunks are sent not in the request body but as a part of the URL query parameters, which feels very retarded.

Not just retarded, but that also applies certain limitation to the chunk size, because URL length has its limits (well, not really, according to RFC, but let’s follow common sense), so setting chunkSize to some big value results in:

Invalid URI: The Uri string is too long.

So I am not sure if sending chunk bytes as a part of query parameters is the way it is supposed to be, but I just couldn’t find the way to send bytes in payload, provide required query parameters and make Twitter to accept all that.

And just look at their example request:

POST https://upload.twitter.com/1.1/media/upload.json?command=APPEND&media_id=123&segment_index=2&media_data=123

They seem to fine with it, so apparently it is okay?

FINALIZE

That is the request you need to send after the last chunk of your file was accepted. You’re letting Twitter know that you finished transferring the file and it should now be processed:

public async Task<Tuple<int, string>> SendFinalize(string mediaID)
{
    using (var httpClient = new HttpClient())
    {
        var finalizeData = new Dictionary<string, string>
        {
            { "command", "FINALIZE" },
            { "media_id", mediaID }
        };

        httpClient.DefaultRequestHeaders.Add(
            "Authorization",
            PrepareOAuth(_TwitterMediaAPI, finalizeData, "POST")
            );
        var httpResponse = await httpClient.PostAsync(
            _TwitterMediaAPI,
            new FormUrlEncodedContent(finalizeData)
            );
        var httpContent = await httpResponse.Content.ReadAsStringAsync();

        return new Tuple<int, string>(
            (int)httpResponse.StatusCode,
            httpContent
            );
    }
}

The response can look like this:

{
  "media_id": 1193135308410889857,
  "media_id_string": "1193135308410889857",
  "media_key": "7_1193135308410889857",
  "size": 165152,
  "expires_after_secs": 86400,
  "processing_info": {
    "state": "pending",
    "check_after_secs": 2
  }
}

If it does not contain processing_info section, then your video is ready to be used, so you can already attach it to a tweet. And if it does contain this section, then it means that the file is being processed and you need to wait until it’s ready.

STATUS

This request allows you to check the processing progress after you sent FINALIZE and got a response with processing_info. If you try to sent STATUS request at any point before that, you’ll get this error:

{
    "errors": [
        {
            "message": "Sorry, that page does not exist",
            "code": 34
        }
    ]
}

And here’s the code for the request:

public async Task<Tuple<int, string>> StatusMedia(string mediaID)
{
    using (var httpClient = new HttpClient())
    {
        var statusData = new Dictionary<string, string>
        {
            { "command", "STATUS" },
            { "media_id", $"{mediaID}" }
        };
        string url = $"{_TwitterMediaAPI}?command=STATUS&media_id={mediaID}";

        httpClient.DefaultRequestHeaders.Add(
            "Authorization",
            PrepareOAuth(_TwitterMediaAPI, statusData, "GET")
            );
        var httpResponse = await httpClient.GetAsync(url);
        var httpContent = await httpResponse.Content.ReadAsStringAsync();

        return new Tuple<int, string>(
            (int)httpResponse.StatusCode,
            httpContent
            );
    }
}

Here’s a response you can get:

{
    "media_id": 1193135308410889857,
    "media_id_string": "1193135308410889857",
    "media_key": "7_1193135308410889857",
    "processing_info": {
        "state": "in_progress",
        "check_after_secs": 1,
        "progress_percent": 37
    }
}

And you guessed it right, you need to run a loop of STATUS requests until the state becomes succeeded (and/or progress_percent becomes 100):

{
    "media_id": 1193135308410889857,
    "media_id_string": "1193135308410889857",
    "media_key": "7_1193135308410889857",
    "size": 165152,
    "expires_after_secs": 86398,
    "video": {
        "video_type": "video\/mp4"
    },
    "processing_info": {
        "state": "succeeded",
        "progress_percent": 100
    }
}

So let’s prepare a helper method:

public int CheckStatus(string mediaID)
{
    var rezStatus = Task.Run(async () =>
    {
        var response = await StatusMedia(mediaID);
        return response;
    });
    if (rezStatus.Result.Item1 == 200)
    {
        var rezStatusJson = JObject.Parse(rezStatus.Result.Item2);
        if (rezStatusJson["processing_info"]["state"].Value<string>() != "succeeded")
        {
            return rezStatusJson["processing_info"]["check_after_secs"].Value<int>();
        }
        else
        {
            return -1;
        }
    }
    else
    {
        throw new Exception($"{rezStatus.Result.Item1}: {rezStatus.Result.Item2}");
    }
}

And now we can combine it with FINALIZE:

var rezFinalize = Task.Run(async () =>
{
    var response = await twitter.SendFinalize(mediaID);
    return response;
});
var rezFinalizeJson = JObject.Parse(rezFinalize.Result.Item2);

if (rezFinalizeJson["processing_info"] != null)
{
    int checkAfter = rezFinalizeJson["processing_info"]["check_after_secs"].Value<int>();
    while (checkAfter != -1)
    {
        Thread.Sleep(checkAfter * 1000); // or some better pause implementation
        try
        {
            checkAfter = twitter.CheckStatus(mediaID);
        }
        catch (Exception ex)
        {
            Console.WriteLine($"[ERROR] {ex.Message}");
            return;
        }
    }
}
else
{
    // your video is ready
}

Posting a tweet with video

Now, when you have an identificator of an uploaded and processed video file, you can attach it to a tweet the same way it was with images:

var rezText = Task.Run(async () =>
{
    var response = await twitter.TweetText("ololo", mediaID);
    return response;
});

For videos to play automatically your users need to enable autoplay in data settings.