KEEP K.I.S.S.

tk's blog

protobuf 跨语言使用字符串编码问题

tk posted @ Dec 24, 2012 02:29:48 PM in C++ with tags protobuf , 22440 阅读

    好久没更新了..... 这段时间,咳

    最近公司项目组有个需求,就是用C++写一个可供 Android 和 WinCE 的跨平台语言底层调用,从下层向上层传递数据。背景是之前项目是在 WinCE 上做的,开发语言是 C#, 现在在开发 Android 的版本,所以想把底层涉密的底层算法用 C++ 重做,同时反馈给之前的 CE 版本。

    跨语言调用传递数据主要就是一个协议,因为不管传递什么数据,底层来说都是一丢丢的字节流,这里只要能从字节流中解析出一致的数据就可以。这里准备采用 google 的 protobuf。

    官方的 protobuf 默认支持三种语言,C++/Java/Python,支持 C# 的有社区版本,protobuf-csharp-port 。

    今天就测试了下从 C++ 传递数据到 C# 中解析的情况,这里暂时没有考虑跨语言直接调用,而是用 C++ 版本 序列化一个文件然后让 C# 版本解析。这之中,如果 .protoc 文件string 定义的字段都是使用 ASCII 编码的话,是没有什么问题的。如果用到中文,这里到 C# 里就会乱码,很常见的编码问题 Orz。主要问题还是 VS 的 C++ 文件编码,因为 VS 默认是 本地编码(中文的就是 GBK),所以里面字符串字面量也是本地编码,如果你把文件编码转换为  UTF-8 无 BOM,会无法编译,如果转换为 UTF-8 带 BOM 格式,VS 编译过程中会把UTF-8带BOM格式文件转换为本地编码.....蛋疼。所以在 C++ 版本里用了一个 GBK 转 UTF-8 的函数,把 GBK 编码的 string 转为 UTF-8 编码的 string,然后序列化。再到 C# 中解析就正常了。如果在 C++版本中不转换而在 C# 中转换编码的话,我找了但是貌似没找到可行的方法。

    无力吐槽 VS 的文件编码处理了。统一向 UTF-8 靠拢才是良策。PS: 貌似 C# 默认写入文件编码就是 UTF-8(待确认)。

    UTF-8 编码避免了字节序(大小端)问题,适合通信。 UTF-16 就很适合本地使用。

    顺便吐槽一下 ParseFromArray 这个 API ,第二个参数必须正好是数组里存放的 message 的 ByteSize 大小,大一点也不行, 所以不能传递数组的大小。原因。所以最好在序列化之前写入 ByteSize 大小用以标识。

  • 无匹配
Liwovosa 说:
May 18, 2021 03:19:17 PM

This is very educational content and written well for a change. It's nice to see that some people still understand how to write a quality post! 123 movies

buy clenbuterol onli 说:
Mar 10, 2023 05:14:01 AM

I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all the details. buy clenbuterol online

SEO 说:
Jun 12, 2023 06:04:01 PM

Thank you for taking the time to publish this information very useful! 프리카지노

sophia 说:
Jun 19, 2023 12:32:51 PM

Most of the time I don’t make comments on websites, but I'd like to say that this article really forced me to do so. Really nice post! 인천출장마사지

sophia 说:
Jun 27, 2023 03:20:33 AM

Thank you again for all the knowledge you distribute,Good post. I was very interested in the article, it's quite inspiring I should admit. I like visiting you site since I always come across interesting articles like this one.Great Job, I greatly appreciate that.Do Keep sharing! Regards, agencia seo

Energy and Utilities 说:
Jul 20, 2023 05:20:05 AM

Tier2Tek staffing is one of the hidden gems of the staffing world. Great rates, amazing quality service.

Sell My Car Denver 说:
Jul 20, 2023 05:21:01 AM

Thank you for sharing this great post! I came across this site while searching for how to sell my car for cash.

Windshield Replaceme 说:
Jul 20, 2023 05:21:58 AM

In regard to this article, we also have the perfect resource for Autoglass services.

Denver web and SEO 说:
Jul 20, 2023 05:22:35 AM

I was looking for SEO services and came across this site. Another great site I recommend is Bus Web Design for SEO experts.

sophia 说:
Aug 21, 2023 01:02:25 AM

This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post. <a href="https://outblasts.com/product/small-rifle-primers-in-stock-now-2021/">small rifle primer</a>

sophia 说:
Aug 21, 2023 01:02:38 AM

This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post. small rifle primer

Real Estate Agent 说:
Aug 28, 2023 06:01:08 AM

If your looking for Online Illinois license plate sticker renewals then you have need to come to the right place.We offer the fastest Illinois license plate sticker renewals in the state. Real Estate Agent

sophia 说:
Aug 29, 2023 03:36:21 PM

Nice blog and absolutely outstanding. You can do something much better but i still say this perfect.Keep trying for the best. เว็บแทงบอลดีที่สุดUFABET

sophia 说:
Aug 31, 2023 09:50:16 PM

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! AVALIADOR PREMIADO PAGA MESMO

sophia 说:
Sep 02, 2023 07:55:06 PM

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! 먹튀검증

sophia 说:
Sep 03, 2023 02:46:16 AM

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! ikaria lean belly juice

sophia 说:
Sep 03, 2023 06:55:27 PM

Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained! Animale Male Enhancement

Dave 说:
Sep 07, 2023 01:26:23 AM

If you are looking for more information about flat rate locksmith Las Vegas check that right away. bubble shooter games

Dave 说:
Sep 17, 2023 12:43:32 AM

Thanks for the blog loaded with so many information. Stopping by your blog helped me to get what I was looking for. car port nz

seo akc 说:
Oct 07, 2023 06:39:05 AM

Nice information, valuable and excellent design, as share good stuff with good ideas and concepts, lots of great information and inspiration, both of which I need, thanks to offer such a helpful information here. <a href="https://www.cbdflex.com/product/30ct-cbd-gummy-bears-300mg/">Buy CBD Gummy Bears</a>

Dave 说:
Oct 09, 2023 01:22:05 PM

This is my first time i visit here and I found so many interesting stuff in your blog especially it's discussion, thank you. LIFT DETOX CAPS

sophia 说:
Oct 11, 2023 06:22:07 PM I definitely enjoying every little bit of it. It is a great website and nice share. I want to thank you. Good job! You guys do a great blog, and have some great contents. Keep up the good work. Fortune Tiger
Dave 说:
Oct 26, 2023 12:38:47 AM

Someone Sometimes with visits your blog regularly and recommended it in my experience to read as well. The way of writing is excellent and also the content is top-notch. Thanks for that insight you provide the readers! Private key recovery

wallpaper singapore 说:
Nov 02, 2023 04:00:31 PM

Thank you very much for writing such an interesting article on this topic. This has really made me think and I hope to read more. Sajta de Pollo

Dave 说:
Nov 13, 2023 09:17:09 PM The packaging is like a puzzle piece; it completes the artistic experience. pencil packing job
Dave 说:
Nov 16, 2023 08:43:44 PM

Cool stuff you have got and you keep update all of us. temazepam 10mg kopen

Dave 说:
Dec 22, 2023 08:43:23 PM

This content is written very well. Your use of formatting when making your points makes your observations very clear and easy to understand. Thank you. roket288

Dave 说:
Dec 24, 2023 06:23:16 PM

Thank you for some other informative website. The place else may just I get that kind of information written in such a perfect method? I have a venture that I am simply now running on, and I’ve been at the glance out for such info. fake personalausweis kaufen

Dave 说:
Jan 07, 2024 03:35:44 PM

hi was just seeing if you minded a comment. i like your website and the thme you picked is super. I will be back. hasch online

seo service london 说:
Jan 16, 2024 02:55:08 PM

The best article I came across a number of years, write something about it on this pag

mubashir 说:
Jan 20, 2024 09:03:46 PM

Listed here you'll learn it is important, them offers the link in an helpful webpage: high-quality

Dave 说:
Feb 09, 2024 01:20:26 PM Please continue this great work and I look forward to more of your awesome blog posts. hasch online
Loto188 说:
Sep 05, 2024 10:18:27 AM

Loto188 la nha cai xo so uy tin nhat Viet Nam. Den nay nha cai da co hon 4 trieu anh em dam me so hoc tham gia choi lo de online va nhieu tro choi khac nhu ca cuoc the thao, casino, no hu. website: http://loto188vn.vip/

Kubet77 说:
Sep 21, 2024 12:44:23 AM

Kubet77 là nhà cái được thành lập từ năm 2005 tại Philippines và trụ sở chính cũng đang đặt tại nước này. Trải qua hành trình dài gần 20 năm, thương hiệu của chúng tôi đã phủ sóng ở nhiều quốc gia thuộc khu vực Châu Á. Nhưng không dừng lại ở đó, Kubet77 vẫn đề ra tầm nhìn chiến lược to lớn đó là mở rộng phạm vi hoạt động đến nhiều đất nước hơn nữa. Chúng tôi hướng đến mục tiêu trở thành sân chơi giải trí số 1 cho khách hàng trên toàn cầu. https://kubet77.market/

Fun88 说:
Nov 06, 2024 09:01:11 AM

Fun88 la thuong hieu nha cai uy tin voi nhung tro choi ca cuoc hap dan hang dau thi truong. Fun 88 duoc bao ho boi Pagcor ngay tu khi ra mat vao nam 2008. Website: https://fun88tc.com


登录 *


loading captcha image...
(输入验证码)
or Ctrl+Enter