This code will indeed convert the return value of rand to a value of type uint8_t. The conversion happens by essentially truncating all but the lowest order 8 bits.

As far as using the modulo operator, most compilers are smart enough to convert modulo by 2ⁿ to a bitwise AND with a value with the n lower order bits set.

Note that for simple uses this is fine, however if you want to generate random numbers suitable for cryptographic purposes, or if you want to ensure better random distribution, you should use a crypto library such as OpenSSL instead of the rand function.

I don't want to use modulo operation, due to time complexity of calculating modulo.

Modern compilers are smart enough to do not use division when it is not absolutely necessary.

All below result in the identical generated code:

uint8_t foo(void)
{
    return rand();
}

uint8_t foo1(void)
{
    return (uint8_t)rand();
}

uint8_t foo2(void)
{
    return rand() % 0x100;
}

uint8_t foo3(void)
{
    return rand() & 0xff;
}

And the generated code:

foo():
        sub     rsp, 8
        call    rand
        add     rsp, 8
        ret
foo1():
        sub     rsp, 8
        call    rand
        add     rsp, 8
        ret
foo2():
        sub     rsp, 8
        call    rand
        add     rsp, 8
        ret
foo3():
        sub     rsp, 8
        call    rand
        add     rsp, 8
        ret

If performance is key, this code would be the fastest way, cause it won't waste 3/4 of the rand() function.

PS rand() function is only guaranteed to return with a 16bit integer, so it might become problematic when your machine is not returning with a 32bit integer.

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

int main(int argc, char *argv[]){
        /*
        typedef struct __attribute__((packed)) _fourbytes{
                uint8_t b0;
                uint8_t b1;
                uint8_t b2;
                uint8_t b3;
        } FOURBYTES;
        */

        typedef union __attribute__((packed)) _doubleword{
                /* FOURBYTES byte; */
                uint32_t dw;
                uint8_t byte[4];
        } DOUBLEWORD;

        DOUBLEWORD random_var;
        random_var.dw = rand();

        for(int i = 0; i < 4; i++){
                printf("random_var.byte[%d] = %d\n", i, random_var.byte[i]);
        }

        return 0;
}