multiple usage of SPEs and DMA   

Hello Everybody,

I'm learning how to develop for CELL, and found some useful articles. At first I studied from kernel.org from geoff examples. Now, I want to use my own program which could transfer data via DMA on multiple SPEs. I wrote my code, but after the SPE computation each SPE cannot write the result into the main memory. Some value is right, but some of them is zero all the time. When I decrease the amount of data, like in the example below (SIZE) the program works perfectly. For example when I want to use the value 768 for the SIZE, it does not work.

PPE-code:

#define SIZE (384)
#define SPE_NUM (6) /* PS3 */
 
main()
{
  /* ... */
  float *in, *out;
  posix_memalign((void **)&in, BYTE_BOUNDARY, SIZE * (sizeof(float)));
  /* ... */
  for (i = 0; i < SPE_NUM; ++i) {
	params[i].in = (unsigned long)&in[i * size];
	params[i].out = (unsigned long)&out[i * size];
	params[i].size = size;
	thread_arg[i].spe = ctx[i];
	thread_arg[i].abs_params = &params[i];
	if ((pthread_create(&thread[i], NULL, run_abs_spe, thread_arg[i])) < 0) {
		perror("pthread_create");
		exit(EXIT_FAILURE);
	}
  }
  /* ... */
  free(in);
  free(out);
  return 0; /* return from main() */
}

SPE-code:

main()
{
        /* ... */
	spu_mfcdma64(&params, mfc_ea2h(argp), mfc_ea2l(argp), sizeof(params),
		     tag, MFC_GET_CMD);
	spu_writech(MFC_WrTagMask, 1 << tag);
	spu_mfcstat(MFC_TAG_UPDATE_ALL);
	/* Read the input data. */
	spu_mfcdma64(in, mfc_ea2h(params.in), mfc_ea2l(params.in), params.size * sizeof(float), tag, MFC_GET_CMD);
	spu_writech(MFC_WrTagMask, 1 << tag);
	spu_mfcstat(MFC_TAG_UPDATE_ALL);
	/* Calculate absolute-value */
	/* ... */
	/* Write the result. */
	spu_mfcdma64(out, mfc_ea2h(params.out), mfc_ea2l(params.out),
		     /*sizeof(params.out) * sizeof(float) */
		     params.size * sizeof(float), tag, MFC_PUT_CMD);
	spu_writech(MFC_WrTagMask, 1 << 2);
	spu_mfcstat(MFC_TAG_UPDATE_ALL);
        */ ... */ 
return 0;
}

I hope this information is enough for you. If not, please ask more!
If you weren't write any convention about this or send an example about it, I'll post here the full code definitely, but firstly it could be confusing. Please let me know place or solutions, how can I solve it on a perfect way?
Thanks for you help,
mAkos

The first thing I noticed in your SPE code:

spu_writech(MFC_WrTagMask, 1 << 2);

Shouldn't it be "tag" instead of "2"? Unless the value of "tag" happens to be 2, of course.

correction:

int size=SIZE/SPE_NUM;

The code you posted works with two minor changes.
You forgot to #include <pthread.h>, and the last argument to pthread_create should be &thread_arg[i] instead of thread_arg[i].

Note, however, that certain values of SIZE will lead to bus errors due to DMA alignment issues.

Actually, it does work for 1152 floats. Each of the six SPEs has to transfer 1152/6 = 192 floats, that is 768 bytes (a multiple of 16 bytes).
Make sure the LS buffers are large enough. Other than that, I see no problem.

As I said, I'm getting the correct results. Can you post any changes you made to your original version?