Dinosaur Technology and Trading

1. ronyboter
   Member

   Hello Karl
   We are trying to get real time quotes for up to 3,000 stocks using RequestMarketData. However IB only allow to stream up to 100 at a time.

   We are trying to find a solution, maybe one that gives us the data in delay of a few minutes. We have tried using the command: "RequestHistoricalData" but the allowed pace is very slow.
   We are trying to keep the delay of data to a minimum for all stocks.

   Is it possible to iterate between the 3,000 stocks, 100 each time using any of the IB functions. SO that every few minutes we have quotes for all 3,000?
   Or, somehow get this using the events?

   Any help would be greatly appreciated.
   Rony Boter

   Posted 3 months ago #

2. Karl
   Key Master

   Hey Rony,

   You should be able to cycle through the Request market data function. I currently cycle every 100 ms. Just create a queue of contracts, as soon as you get data, dequeue, cancel the market data, and then request the next contract in the list. I am not sure exactly how fast you can do this though, as IB has a 50 outbound message/second limitation, so at a minimum it would be 3,000/(50/2) (place and cancel message required, hence the two). So that would be 120 seconds to cycle through.

   -Karl

   Posted 3 months ago #

3. Neil
   Member

   Hi Karl, I love the API, thank you very much for making it. I have a question.

   In between your data requests and your cancel data requests, do you have code that waits the main thread so that the data request is given enough time to get the data before the main thread calls cancel on it?

   I tried running this code:

   client.RequestMarketData(NextTickerId, equity, null, true);
   client.CancelMarketData(NextTickerId++);

   But the main thread cancels the market data before the first function has time to get the data and print it to the console.

   I have this as a crude solution:

   client.RequestMarketData(NextTickerId, equity, null, true);
   Console.Readline();
   client.CancelMarketData(NextTickerId++);

   With this code, the data is requested and printed to the console, and then I press enter, and then the data is cancelled.

   Do you have a method for solving this issue, for having the main thread wait for the reader thread to get the data before the main thread calls cancel on it?

   Posted 3 months ago #

4. Karl
   Key Master

   The easiest way is to use something like a semaphore. Put a waitone call in between the request and cancel, and have the market data event generate a release after updating the variable.

   Posted 3 months ago #

5. Neil
   Member

   I figured it out by using an AutoResetEvent which is similar to a Semaphore. Here is code of an AutoResetEvent being used to wait the main thread until the reader thread records an ask price in a variable before the main thread submits a limit order at that ask price and then cancels the previously-made data request:

   // Global scope:
   private static AutoResetEvent got_ask_event = new AutoResetEvent(false);
   private static Decimal ask_price;

   // Main():
   Equity RL = new Equity("RL");
   client.RequestMarketData(NextTickerId, RL, null, true);
   Order opening_order = new Order();
   opening_order.Action = ActionSide.Buy;
   opening_order.OrderType = OrderType.Limit;
   opening_order.TotalQuantity = 100;
   got_ask_event.WaitOne(); // Waits for got_ask_event.Set()
   opening_order.LimitPrice = ask_price;
   client.PlaceOrder(NextOrderId++, RL, opening_order);
   client.CancelMarketData(NextTickerId++);

   // Event handler run by the reader thread:
   static void client_TickPrice(object sender, TickPriceEventArgs e)
   {
   switch (e.TickType)
   {
   case TickType.AskPrice:
   {
   ask_price = e.Price;
   got_ask_event.Set(); // Got the ask, now free the main thread
   break;
   }
   }
   }

   Posted 3 months ago #

6. Neil
   Member

   Is there a way to format code as to appear well-spaced on this forum?

   Yes there is: "code" in angle brackets or backticks.
   It appears though that this markup is broken and breaks when you edit your post and resubmit the same post as before, the code disappears and a single character replaces it.

   


   M

   Posted 3 months ago #

7. Karl
   Key Master

   Hello Neil - I use bbPress for the forums, so I am not sure if there is a way to do code blocks. If it isn't built in, I'll take a look around for some addins...

   Thanks for the response using the AutoResetEvent - looks good!

   Posted 3 months ago #

8. ronyboter
   Member

   Dear Karl,
   I tried Neil's solution,
   But I'm still getting the error that I reached the maximum.
   Here is my code, got in the list about 3000 tickers.

   for (i = 1; i < companiesNew.Count; i++)
   {
   G2List<DDCompanyEquity> companiesNew1 = ConvertToG2List<DDCompanyEquity>(companiesNew.GetRange(i, 1));

   client.RequestCompanyEquitiesMarketData(companiesNew1);
   got_ask_event.WaitOne();
   }

   static void client_TickPrice(object sender, TickPriceEventArgs e)
   {

   switch (e.TickType)
   {
   case TickType.AskPrice:
   {
   ask_price = e.Price;
   client.CancelMarketData(e.TickerId);
   got_ask_event.Set(); // Got the ask, now free the main thread
   break;
   }
   }
   Console.WriteLine("Price: " + e.Price + " Tick Type: " + EnumDescConverter.GetEnumDescription(e.TickType));
   }

   What is wrong I still get those errors, is there any solution to get real-time data as continuously?
   Many thanks
   Notes:
   I tried your solution as well (with semaphores thought they are quite the same).
   RequestCompanyEquitiesMarketData is wrapper for your function.
   Any help one this would be greatly appreciated.

   Rony

   Posted 2 months ago #

9. ronyboter
   Member

   We have tried looping 3000 tickers, each time requesting data for just one ticker(Request.MarketPrice) and then canceling the request. This works but takes about 30 minutes.

   So now we are trying to loop batches of 50 tickers on each request, this works partially. Some of the data is received and some has errors.
   We get the following errors:
   1) Requested market data is not subscribed.Error&BEST/STK/Top

   2) Can't find EId with tickerId:206

   3) Unable to write data to the transport connection: An established connection was aborted by the software in your host machine.

   Code:
   for (i = 0; i < ((int)companiesNew.Count/MAX_TICKERS); i++)
   {

   G2List<DDCompanyEquity> companiesNew1 = ConvertToG2List<DDCompanyEquity>(companiesNew.GetRange(i*MAX_TICKERS, MAX_TICKERS));
   client.RequestCompanyEquitiesMarketData(companiesNew1);
   got_ask_event.WaitOne(new TimeSpan(0, 0, 30), false);
   client.CancelMarketData(i, i * MAX_TICKERS);
   Thread.Sleep(new TimeSpan(0, 0, 10));
   initIndicators();
   }
   TimeSpan times = DateTime.Now - dtfrom;
   Console.WriteLine("IT TOOK "+ times.Seconds.ToString());
   Console.ReadLine();
   Thread.Sleep(100);
   }

   static void client_Error(object sender, ErrorEventArgs e)
   {
   Console.WriteLine("Error: " + e.ErrorMsg );
   if (((int)e.ErrorCode) == 200)
   {

   RealTimeIndicators[e.TickerId%MAX_TICKERS] = true;
   }

   }

   static void client_TickPrice(object sender, TickPriceEventArgs e)
   {

   switch (e.TickType)
   {
   case TickType.AskPrice:
   {
   ask_price = e.Price;
   // client.CancelHistoricalData(e.TickerId);
   // client.CancelMarketData(e.TickerId);
   // got_ask_event.Set();
   //// client.CancelRealTimeBars(e.TickerId);
   Console.WriteLine("Price: " + e.Price + " Tick Type: " + EnumDescConverter.GetEnumDescription(e.TickType)+" on ticker id :"+e.TickerId.ToString());
   RealTimeIndicators[e.TickerId % MAX_TICKERS] = true;
   if (CheckIndicators())
   {
   client.CancelMarketData(e.TickerId-MAX_TICKERS, e.TickerId );
   got_ask_event.Set();
   }

   // Got the ask, now free the main thread
   break;
   }
   }

   So far we have not found the cause of the errors, any help would be greatly appreciated.
   Thanks for all the support.

   Posted 2 months ago #

RSS feed for this topic

Reply

You must log in to post.