It seems that a library loaded using JNI cannot load other C libraries on run-time because we are in a Java environment. Is my assumption correct?

No.

How is libsecond.so used by libfirst.so? Is it a linked dependency, or is it loaded by dlopen?

I've found that something like:

static {
    System.loadLibrary("second");
    System.loadLibrary("first");
}

in the class that uses the JNI usually works.

EDIT: Now that I know how you are loading libsecond.so this worked for me:

Test.java

public class Test {

    public static void main (String args[]) {
        test();
    }

    private native static void test();

    static {
        System.loadLibrary("first");
    }
} 

first.c -- The sole translation unit of libfirst.so

#include <jni.h>
#include "Test.h"
#include <dlfcn.h>
#define LIBNAME "libsecond.so"

#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     Test
 * Method:    test
 * Signature: ()V
 */
JNIEXPORT void JNICALL
Java_Test_test(JNIEnv *env , jclass cls)
{
  void* h;
  void (*sym)(void);

  h = dlopen(LIBNAME, RTLD_LAZY|RTLD_GLOBAL);
  if (h) {
    printf("dlopen " LIBNAME " worked\n");
    sym = (void (*)(void))dlsym(h,"second");
    sym();
  } else {
    printf("dlopen " LIBNAME " failed\n");
  }
}

#ifdef __cplusplus
}
#endif

second.c -- The sole translation unit of libsecond.so

#include <stdio.h>

void
second(void)
{
  printf("hello from second\n");
}

Makefile

CFLAGS=-fPIC

all : libfirst.so libsecond.so 

libsecond.so : second.o
        $(CC) -shared -Wl,-soname,libsecond.so.0 -o $@ $^ -lc

libfirst.so : first.o
        $(CC) -shared -Wl,-soname,libfirst.so.0 -o $@ $^ -ldl -lc

clean:
        rm -f *.o *.so

Test.h can be produced by javah Test. Note that libfirst.so and libsecond.so are not linked together.

All usual rules apply to dopen() from a library loaded from Java. Particularly, you check your LD_LIBRARY_PATH, rpath, runpath, etc. See also dlopen failed：cannot open shared object file: No such file or directory