I have a Django app that spawns a subprocess everytime there is a database insert.

models.py

    # spawn subprocess to trigger tweepy
# output of subprocess DOES NOT log to the console.
def tweepy_tester(sender, **kwargs):
    if kwargs['created']:
        logger.error('tweepy trigger-start!')
        p = subprocess.Popen([sys.executable, "/Users/viseshprasad/PycharmProjects/Blood_e_Merry/loginsignup/tests.py"],
                             stdout=subprocess.PIPE,
                             stderr=subprocess.STDOUT)
        logger.error('tweepy trigger-over!')


# use post_save to trigger tweepy later
post_save.connect(tweepy_tester, sender=User)

tests.py

logger = logging.getLogger(__name__)


# Create your tests here.

def for_thread():
    i = 0
    while True:
        f = open('test.txt', 'a')
        f.write('Tweepy triggered ' + str(i) + '\n')  # python will convert \n to os.linesep
        f.close()  # you can omit in most cases as the destructor will call it
        i += 1


for_thread()

The trigger happens fine but the subprocess only writes 3640 lines to the test.txt file, even though I have used while True: I am basically look for a subprocess to run non-stop after the trigger, as a separate thread and not disturbing the main thread.

The purpose : I run my app with the usual python manage.py runserver. User signs-up -> database insert -> triggers my implementation of tweepy which keeps on streaming tweets and analyzing them non-stop on a different background thread so as to not interfere with the signup process.

The above test is for this purpose. Any help is appreciated. Any alternative suggestions to implement this are also welcome.

Thanks.